FlashInfer patches unlock native FP4 on Blackwell

This is exactly the kind of low-level inference bug that can make a brand-new GPU generation look broken until somebody digs through the kernel stack by hand.

• –The biggest takeaway is correctness, not speed: the default grouped GEMM path was emitting incoherent tokens on SM120 desktop Blackwell, so “native FP4 support” was not actually usable for MoE workloads.
• –The working path runs through FlashInfer CUTLASS with SM120 capability patches and CUDA 13.0’s `compute_120f`, which strongly suggests architecture targeting and TMA kernel enablement were the real bottlenecks.
• –39 tok/s still trails Marlin’s 46-49 tok/s, but it is close enough to prove native NVFP4 on desktop Blackwell is viable once upstream kernels stop falling back to slow tactics.
• –For AI infra teams deploying Blackwell workstations, this is a useful warning that feature-matrix support and real-world kernel behavior can diverge badly on new architectures.