Mistral Small 4 posts strong single-GPU benchmark

This is a solid real-world datapoint for teams considering a single-card Mistral Small 4 deployment: throughput is usable, but responsiveness depends heavily on prompt-caching strategy.

• –The benchmark confirms worst-case uncached behavior, so production chat/coding workflows with incremental context should see materially better TTFT.
• –At 32K context, capacity lands around 3 concurrent requests under conservative UX thresholds, which is practical for small internal copilots.
• –At 64K-96K, TTFT becomes the binding constraint before decode speed, so queueing and admission control matter more than raw tok/s.
• –The reported vLLM-vs-SGLang tradeoff (better TTFT vs slower decode) suggests stack tuning is now as important as model choice on Blackwell-class cards.
• –For long-context agent workloads, KV-cache optimization and speculative decoding support will likely determine whether this setup scales beyond single-user “power mode.”