OPEN_SOURCE ↗
REDDIT · REDDIT// 11d agoOPENSOURCE RELEASE
SwiftLM adds TurboQuant, SSD expert streaming
SwiftLM is a native Swift MLX inference stack for Apple Silicon that pairs TurboQuant KV compression with SSD-backed expert streaming for large MoE models. The same codebase also ships an iPhone app that runs smaller Qwen3 models on-device.
// ANALYSIS
This is a strong systems-first local AI project: it attacks the two real bottlenecks, KV cache growth and MoE weight residency, instead of just squeezing another quantization ratio out of the model. The claims are ambitious, but the architecture is credible enough that the runtime numbers are the part worth watching.
- –TurboQuant matters because KV dequant overhead is usually where clever compression schemes die; fusing it into Metal is the right place to pay that cost.
- –SSD expert streaming is a pragmatic answer to oversized MoE models on macOS, especially if the OS page cache can keep hot experts warm without manual orchestration.
- –The iPhone angle is narrower but real: on-device Qwen3 for 0.6B/1.7B classes is useful, even if it does not mean full-sized frontier models fit comfortably.
- –Open-source implementation detail will matter more than the headline performance numbers; this kind of stack tends to win or lose on edge cases, not demo runs.
- –The project sits in the sweet spot between inference infrastructure and end-user apps, which makes it unusually relevant for Apple-platform AI builders.
// TAGS
swiftlmmlxinferencegpuedge-aiopen-sourceai-coding
DISCOVERED
11d ago
2026-04-01
PUBLISHED
11d ago
2026-04-01
RELEVANCE
9/ 10
AUTHOR
solderzzc