Gemma 4 32B Trips TensorRT-LLM Setup

This is less a launch story than a real-world deployment check: the fastest inference stack on paper is often the one with the sharpest setup edge cases.

• –TensorRT-LLM still looks powerful, but model conversion and deployment friction can erase the theoretical gains for newer models like Gemma 4 32B
• –vLLM remains the practical baseline because it is easier to get running, even when it is not the absolute fastest
• –Modular MAX becoming usable in the thread is a reminder that newer serving stacks are still proving themselves in day-to-day workflows
• –The RTX 6000 Pro angle matters: high-end hardware does not eliminate compatibility and tooling issues
• –This is useful signal for anyone benchmarking serving engines, because “works eventually” is not the same as “works reliably”