Gemma 4 MTP Drafter Doubles H100 Throughput

This is the kind of optimization that matters more than raw parameter-count bragging: speculative decoding turns idle GPU time into throughput, and Google is packaging it in a way ordinary Transformers users can actually try.

• –The key win is architectural, not magical: the drafter proposes tokens, the main model verifies them, and the final output stays identical to baseline inference.
• –The setup friction looks low, which is important for adoption. If it really is just a couple of Python lines in Transformers, teams can test it without replatforming to vLLM or a special serving stack.
• –DFlash and MTP are solving a similar bottleneck with different mechanics, so the real choice is likely operational simplicity versus whatever performance edge the more specialized stack can squeeze out.
• –Treat the posted benchmark as directional, not universal. The author already notes a dtype/config caveat, so the exact 2x number may move around once the setup is tightened.
• –For anyone running large local or cloud Gemma workloads, this is a strong sign that inference efficiency is becoming a first-class feature of the model release, not just an afterthought.