Qwen3.8-27B Runs at 417 Tok/s on 3090

This is a strong proof point for local inference: consumer GPUs can now serve capable 27B models at genuinely useful speeds, provided developers are willing to manage a specialized stack.

• –Quantizing the embedding table and lm_head is the key memory-saving trick, freeing VRAM for context and batching.
• –vLLM patches, CUDA optimizations, and separate batch versus single-user configurations make this a deployment recipe rather than a plug-and-play model download.
• –The headline throughput is aggregate batched performance; single-user latency will be substantially lower, so developers should benchmark against their own concurrency and prompt lengths.
• –Qwen officially lists the model as a 27B multimodal open-weight release with native 262K context, widening its usefulness for coding agents and document-heavy workflows. [Official Qwen repository](https://github.com/QwenLM/Qwen3.8)