OPEN LINK ↗
// 2h agoOPENSOURCE RELEASE
TurboFieldfare runs 26B Gemma 4 on 2GB Macs
TurboFieldfare is an open-source inference engine written in Swift and Metal that makes it possible to run the 4-bit quantized Gemma 4 26B-A4B-IT model on any Apple M-series Mac with as little as 2 GB of available RAM. By dynamically streaming routed experts from the SSD while keeping the shared core in memory, it overcomes hardware limits to achieve generation speeds of 5-6 tokens per second on an 8GB M2 MacBook Air.
// ANALYSIS
Running a 26B parameter model in just 2 GB of RAM is an impressive feat of systems engineering that significantly lowers the barrier to entry for local AI inference.
- –Utilizes SSD streaming and an expert cache alongside bounded parallel `pread` to fetch weights on demand while the GPU processes shared layers.
- –Achieves very usable speeds (31-35 tok/s on an M5 Pro) despite relying on slower SSD storage.
- –Democratizes access to larger open-source models for users on base-model Macs with 8GB or 16GB of unified memory.
- –The inclusion of an OpenAI-compatible server makes it an immediate drop-in replacement for cloud APIs in local development workflows.
// TAGS
aillmopen-sourcemacosinferenceswiftmetal
DISCOVERED
2h ago
2026-07-29
PUBLISHED
3h ago
2026-07-29
RELEVANCE
9/ 10
AUTHOR
gitpusher42