llama-server tweaks boost local LLM serving speed
A LocalLLaMA user shared optimization tips for running llama-server on systems with limited VRAM. Running with the -np 1 flag and disabling browser hardware acceleration frees up VRAM to significantly boost tokens-per-second output.
Simple configuration tweaks can dramatically improve local LLM inference speeds on consumer hardware by efficiently managing limited VRAM resources. The llama-server defaults to allocating context for multiple concurrent clients, but using the -np 1 flag reserves VRAM strictly for a single user. Furthermore, disabling hardware acceleration in background applications like Firefox releases chunks of VRAM that the OS otherwise reserves for web page rendering. Combined, these adjustments can yield over a 20% increase in tokens per second, allowing users to run larger quantizations or achieve higher throughput on mainstream GPUs.
DISCOVERED
16d ago
2026-03-26
PUBLISHED
16d ago
2026-03-26
RELEVANCE
AUTHOR
ea_man