llmBench maps local GPU perf to Arena leaderboard
llmBench is an open-source Python tool that benchmarks local LLM inference on Ollama and llama.cpp, then maps your hardware's performance against the LMSYS Chatbot Arena leaderboard. It also analyzes VRAM/RAM to recommend which models your rig can run efficiently.
The gap between "can I run it?" and "how good is it globally?" has been a real frustration for local LLM enthusiasts — llmBench is a direct attempt to bridge both questions in one tool.
- –Unique angle: maps local tokens/sec and VRAM metrics against the LMSYS Arena leaderboard, giving consumer hardware a global performance frame of reference that tools like LocalScore don't provide
- –Tracks energy efficiency (Joules per token) and thermal behavior alongside standard throughput — useful for laptop users running on constrained TDPs
- –Hardware forensic mode digs into PCIe bandwidth, RAM manufacturer, and DDR generation — surfacing hidden bottlenecks beyond just VRAM size
- –Currently Windows-only (WMI-dependent) and requires an NVIDIA GPU with nvidia-smi, limiting the audience
- –Very early stage (2 GitHub stars, 8 commits) — promising concept but not yet battle-tested or cross-platform
DISCOVERED
75d ago
2026-03-15
PUBLISHED
75d ago
2026-03-15
RELEVANCE
AUTHOR
Cod3Conjurer
