M5 Max prefill gains peak near 16K prompts

Apple's 4x claim is real, but it's a narrow win: the chip is being tested in the exact phase where Neural Accelerators matter most.

• –Apple’s own MacBook Pro page pegs M5 Max at 6.7x faster TTFT than M1 Max, but only 1.7x faster than M4 Max, which shows how much of the magic depends on baseline and workload.
• –Apple’s MLX research on the M5 family says first-token generation is compute-bound, while subsequent tokens only improve 19-27% because they’re memory-bandwidth-bound.
• –That makes M5 Max especially compelling for local coding assistants, RAG-heavy prompts, and other workflows where latency spikes happen before the model ever starts streaming.
• –The 16K-token benchmark is the tell: it’s long enough to stress prefill, but it’s still measuring the best-case phase of inference.
• –Practical takeaway: benchmark TTFT and generation separately, or the same chip can look either miraculous or merely incremental.