Gemma 4 tops closed chats

This is not a rigorous benchmark, but it is exactly the kind of workflow-specific eval that matters to developers more than leaderboard averages.

• –Gemma 4 31B is an open-weight dense model from Google DeepMind with multilingual support, a 256K context window, and local deployment paths through Hugging Face and other runtimes.
• –Translation with hidden identities and name consistency stresses long-context tracking, entity resolution, and style control, where small regressions are very visible to users.
• –The complaint about closed-model A/B testing is the real punchline: if a provider silently changes routing or safety behavior, users can lose a working workflow overnight.
• –The result also cuts against the assumption that Gemini Chat must expose Google’s best Gemma-like behavior; consumer chat products are shaped by routing, guardrails, cost, and UX constraints.
• –Treat this as a strong prompt-level data point, not proof that Gemma 4 beats frontier closed models generally.