Deltafin runs 2.8T Kimi K3 MoE on Apple Silicon
Deltafin is an open-source project by Gavamedia that allows users to execute Kimi K3, a massive 2.8-trillion-parameter Mixture-of-Experts (MoE) model, on consumer Apple Silicon hardware. By dynamically streaming required expert weights over HTTP into a local disk cache with Metal optimizations, it achieves local inference and provides an OpenAI-compatible API endpoint.
Dynamic expert streaming for MoE models is a game-changer that proves clever software engineering can bypass strict hardware VRAM limits for local AI.
* Enables local execution of 2.8T parameter models on consumer Apple Silicon hardware.
* Bypasses memory constraints by streaming only necessary MoE expert weights into a local cache.
* Leverages custom MPS and Metal kernels to achieve ~16 seconds per token generation on an M1 Max.
* Includes an OpenAI-compatible API server for immediate UI and agent compatibility.
DISCOVERED
1h ago
2026-07-29
PUBLISHED
4h ago
2026-07-28
RELEVANCE
AUTHOR
tito