Isotropic Tradeoff exposes rotation's sparsity tax

Provocative framing aside, the useful takeaway is that rotation-based quantization may optimize the wrong proxy if your real concern is preserving sparse internal structure.

• –TurboQuant, QuaRot, and QuIP all try to smooth outlier channels so low-bit scalar quantization behaves better; this project argues the hidden cost is semantic sparsity loss, not just numerical distortion.
• –The metrics move in opposite directions: outlier reconstruction and cosine similarity improve, while the quiet end of the activation distribution gets noisier.
• –The big unanswered question is downstream effect. If ghost activations hurt reasoning or calibration, this is a real eval gap; if not, it is mostly a representational artifact.
• –The notebook, fixed seeds, and published draft make it easy to rerun, but the claim still needs broader cross-model replication before it becomes a general indictment.