OPEN LINK ↗
// 63d agoBENCHMARK RESULT
RTX 5090 tops speed, AMD wins capacity
New benchmarks compare the RTX 5090 against AMD’s AI395 (Strix Halo) and dual R9700 setups, revealing that while NVIDIA leads in raw speed for smaller models, AMD’s unified memory architecture is essential for running 122B+ parameter models locally.
// ANALYSIS
VRAM capacity, not raw compute, is now the primary bottleneck for local LLM performance.
- –RTX 5090 delivers unmatched throughput (205 t/s) for models under 32GB but is effectively barred from 70B+ dense models.
- –AMD AI395 (Strix Halo) proves the value of unified memory, running 122B MoE models at 20 t/s—a feat impossible on single consumer NVIDIA cards.
- –ROCm dominates prompt processing performance, but Vulkan surprisingly yields better generation speeds for MoE architectures.
- –Dual-GPU setups on AMD show significant prompt processing gains but highlights interconnect overhead as generation speeds remain flat.
- –Disabling mmap (`-mmp 0`) is a critical optimization for APUs to prevent iGPU choking on large shared-memory allocations.
// TAGS
llama-cppbenchmarkgpuinferenceopen-sourcellm
DISCOVERED
63d ago
2026-03-25
PUBLISHED
63d ago
2026-03-25
RELEVANCE
8/ 10
AUTHOR
ReasonableDuty5319
