DeepSeek R1 hits MI300X with free inference test

This public test is a major milestone for the AMD ROCm ecosystem, proving that the MI300X's 192GB VRAM can efficiently handle massive 671B parameter models like DeepSeek R1 on a single node.

• –MI300X's high memory bandwidth (5.3 TB/s) is uniquely suited for the Multi-head Latent Attention (MLA) and Mixture-of-Experts (MoE) architecture used in DeepSeek R1.
• –Initial performance metrics of 45 tokens/second per user suggest AMD is becoming a viable and cost-effective alternative to NVIDIA H100/H200 for production inference.
• –Zyphra's specific focus on long-context (32k token output) testing leverages the VRAM advantage of AMD hardware, which is critical for complex agentic workloads.
• –The successful deployment of frontier models by labs like Zyphra signals a broader industry shift toward compute diversification and native ROCm optimization.