Calculator compiles into transformer weights

This is a clever proof-of-concept, not a practical calculator. The interesting part is the mental model shift: attention becomes routing, residuals become state, and the transformer starts looking more like a programmable machine than a fuzzy text predictor.

• –Attention weights are computed by the compiler, so the routing logic is explicit and deterministic rather than learned
• –The MLPs are trained layer-by-layer to mimic exact Python logic, which keeps the system faithful but exposes how hard some operations are to learn directly
• –The register and liveness analysis framing is the strongest idea here, because it makes depth look like a dependency graph with reusable storage
• –The article’s main limitation is also its main research question: can the MLP weights eventually be constructed directly instead of distilled
• –For AI devs, the takeaway is less “build a transformer calculator” and more “transformers can be engineered as structured computation substrates”