Qwen 3.5 MoE tops Gemma 4 M5 benchmarks

The M5 Max and oMLX are turning local Macs into viable high-performance inference servers, with MoE architectures clearly winning on Apple Silicon.

• –Qwen 3.5 MoE (35B-A3B) is the current performance champion, achieving 92.2 tok/s for generation and nearly 2,850 tok/s for prompt processing.
• –oMLX's tiered KV caching leverages SSD storage to restore context prefixes in under 2 seconds, a massive improvement over the 60+ second prefill times seen in standard MLX implementations.
• –The M5's Neural Accelerator specifically boosts the prefill stage, making dense models more responsive but not yet competitive with MoE throughput.
• –While Gemma 4 is more memory-efficient and responsive for "edge" tasks, it lags behind Qwen in sustained batching and serving performance for heavy developer workloads.
• –SSD-based context persistence is becoming the new baseline for "agentic" local LLM tools like Claude Code and Cursor.