PyTorch 2.9 ships Muon optimizer

Muon is interesting, but it reads like an optimizer you benchmark carefully, not one you casually flip on for every fine-tune.

• –PyTorch’s docs say Muon is meant for 2D hidden-layer parameters; non-2D params still belong on AdamW, so parameter grouping is the first real hurdle.
• –The 2.9 optimizer table lists Muon as `for-loop` only, with no foreach or fused implementation, which suggests its gains are algorithmic rather than kernel-level: https://docs.pytorch.org/docs/2.9/optim.html
• –For VRAM-constrained training, the appeal is optimizer-state efficiency, not a drop-in replacement for AdamW.
• –The Reddit thread is still empty, which fits the current vibe: curious, promising, but not yet battle-tested in the local fine-tuning crowd: https://www.reddit.com/r/LocalLLaMA/comments/1rxe7jl/torchoptimmuon_is_now_in_pytorch_29_anyone/