Orion cracks Apple Neural Engine training

This is a genuinely interesting systems release because it turns Apple’s heavily abstracted NPU from an inference-only black box into something researchers can actually program against. Orion is less about raw training speed today than proving that low-level, on-device LLM training on ANE is possible at all.

• –Orion bypasses CoreML with direct ANE execution, a custom graph IR, and compiler passes that emit ANE-native MIL
• –The standout claim is stability: the project reports 1,000 training steps on a 110M transformer with loss dropping from 12.29 to 6.19 and zero NaN divergence
• –Delta compilation and LoRA hot-swap make the repo more than a one-off hack; it is shaping into a reusable local AI runtime for Apple Silicon
• –The bottleneck is still brutal because ANE weight updates are constrained by compile and reload behavior, so this is a platform breakthrough before it is a practical fine-tuning stack
• –For AI infra and edge researchers, Orion is a strong signal that Apple’s Neural Engine has far more untapped developer value than the official ML stack exposes