Gemma 4 26B A4B Hits 40K Context

This is a strong systems demo: it shows long-context LLMs are often memory-management problems as much as model problems.

• –Block-level KV compression is the right tradeoff here because it preserves recent context quality while cutting VRAM pressure from older tokens
• –The reported stability matters more than the raw 40k number; a flat memory plateau and no OOMs are what make the setup usable
• –The implementation reads like a practical vLLM fork, not a kernel research project, which makes it more relevant for real deployments
• –Limits still matter: no batch optimization, periodic restarts, and a validated safe range mean this is not yet a general-purpose long-context solution