OPEN LINK ↗
// 1h agoOPENSOURCE RELEASE
Swiftlet runs 80B MoE models on iPhones
Swiftlet is an open-source Swift and Metal inference runtime designed to execute 35B and 80B Qwen Mixture-of-Experts (MoE) models on Apple hardware by streaming routed expert parameters directly from SSD. By keeping only the dense core model in RAM and using custom QPack binary files with Metal kernels, Swiftlet minimizes RAM footprint down to 2.5GB while enabling local LLM inference on memory-constrained consumer devices like iPhones.
// ANALYSIS
Dynamically streaming MoE expert weights from fast storage is a major practical breakthrough for edge AI on memory-constrained consumer hardware.
- –Bypasses traditional mobile RAM limitations by offloading non-active MoE expert parameters to high-speed SSD storage.
- –Leverages Metal compute kernels and custom QPack file formats optimized for fast parameter loading and execution.
- –Enables privacy-preserving, local execution of 35B and 80B scale models on Apple iPhones without requiring cloud infrastructure.
// TAGS
swiftletappleiosmetalqwenmoellmlocal-aiswiftopen-source
DISCOVERED
1h ago
2026-08-04
PUBLISHED
1h ago
2026-08-04
RELEVANCE
8/ 10
AUTHOR
Github Awesome