Midjourney founder: diffusion wins as FLOPS outpace memory

Holz is betting on the "Memory Wall" to consolidate the industry around diffusion, framing autoregression as a legacy architectural choice for a world where bandwidth was abundant.

• –Scaling FLOPS is physically easier than scaling memory bandwidth, making compute-heavy diffusion more future-proof.
• –Autoregressive models (LLMs) suffer from high "read" costs per token, leaving massive GPU compute capacity idle during generation.
• –Diffusion’s iterative denoising process is "all math," allowing it to eat up the massive FLOPS increases in next-gen AI accelerators.
• –This suggests a future where logic is learned via autoregression but the world is "rendered" and inferred via diffusion.
• –Midjourney’s refusal to pivot to Visual Autoregressive (VAR) models now looks like a strategic hardware play rather than just an aesthetic preference.