Qwen3.6 27B speeds split local users

// 45d agoBENCHMARK RESULT

Qwen3.6 27B speeds split local users

LocalLLaMA users are comparing Qwen3.6-27B throughput after one llama.cpp setup reported about 13 tokens/sec on Q8_0 with 128K context across mixed RTX 2060 Super and 5060 Ti GPUs. The thread shows a wide spread, from similar llama.cpp results on consumer cards to much higher numbers on RTX 5090 and vLLM/MTP setups.

// ANALYSIS

The useful signal here is not that one rig is “slow,” but that Qwen3.6-27B makes inference-stack choices brutally visible: context length, quant, KV cache, split mode, and serving engine all matter.

–Qwen3.6-27B is a 27B open-weight model with vision support and a native 262K context window, so 128K context plus Q8 cache is a heavy local-serving configuration.
–llama.cpp users are reporting materially different speeds depending on GPU mix, quantization, flash attention, split strategy, and whether the model stays fully in VRAM.
–vLLM and MTP-backed runs appear much faster in the thread, reinforcing Qwen’s own guidance that production throughput favors engines like vLLM, SGLang, or KTransformers.
–For developers, the practical takeaway is to benchmark at the actual context length and modality settings they plan to use, not just compare headline tokens/sec.

// TAGS

qwen3.6-27bllminferencegpuopen-weightsself-hostedbenchmark

DISCOVERED

45d ago

2026-04-22

PUBLISHED

45d ago

2026-04-22

RELEVANCE

8/ 10

AUTHOR

Ambitious_Fold_2874

// KEEP READING

More AI developer news from the feed

EXPLORE FULL FEED

LAUNCH12m ago

NVIDIA announces the Arm-based RTX Spark superchip combining Grace CPU and Blackwell RTX GPU to run high-performance AI agents locally on Windows PCs.

NVIDIA has introduced the RTX Spark, an Arm-based system-on-chip (SoC) combining a Grace CPU and Blackwell-architecture RTX GPU with up to 128GB of unified memory. Designed to run high-performance AI agents locally on Windows PCs, the superchip delivers up to 1 petaflop of FP4 AI performance, enabling consumers to run large language models of up to 120 billion parameters without relying on cloud infrastructure.

RESEARCH12m ago

Stability AI releases Stable-Layers, a reinforcement learning framework that trains image layer decomposition models without paired supervision data using Flow-GRPO and VLM feedback.

Stability AI has introduced Stable-Layers, a reinforcement learning framework designed to train image layer decomposition models without requiring paired training datasets. Traditionally, splitting a flat image into editable, multi-layer components required intensive human annotation. Stable-Layers bypasses this by adapting Group Relative Policy Optimization (GRPO) for flow-matching models (Flow-GRPO) to optimize image decomposition. The training is guided by a Vision-Language Model (VLM) serving as a judge, using a structured scoring and grid-based calibration pipeline to provide high-quality reward signals. This approach significantly reduces color bleed and blank layer artifacts, producing cleaner semantic separation.

MODEL12m ago

Boson AI launches Higgs Audio v3

Boson AI has launched Higgs Audio v3, a 4B parameter text-to-speech model built on a Qwen3-4B backbone and optimized for real-time conversational streaming and zero-shot cloning across 100+ languages. The model supports inline style tags for prosody and emotion control, and integrates with the newly released SGLang-Omni inference framework for low-latency deployment.

Qwen3.6 27B speeds split local users