Clip to Grok hits 249x speedup

Weight norm clipping is the "hard" regularization that weight decay always wanted to be.

• –Replaces slow, "soft" weight decay with a rigorous per-row L2 clipping that forces models into the "generalization zone" instantly.
• –Dramatically reduces "grokking delay" by preventing models from staying in high-norm memorization regimes.
• –Implementation is trivial (few lines of PyTorch) and has already been integrated by community stalwarts like lucidrains in fast-weight-attention.
• –Shows massive synergy with sign-based optimizers like Lion, suggesting a new primitive for fast-generalizing training loops.
• –Findings reveal that optimal max_norm correlates with algebraic complexity, with non-abelian tasks requiring tighter constraints (1.0) than modular addition (2.0).