OPEN LINK ↗
// 1h agoOPENSOURCE RELEASE
ESP32 AI Runs 28.9M Model at 9.5 Tokens/Sec
ESP32 AI is an architectural experiment by slvDev that runs a 28.9-million-parameter TinyStories language model locally on an $8 ESP32-S3 microchip without relying on external cloud servers. By keeping a 25-million-parameter embedding table in memory-mapped SPI flash to fetch token rows on demand, the project successfully circumvents tight microcontroller RAM limitations while maintaining a generation throughput of approximately 9.5 tokens per second.
// ANALYSIS
Proving that local language model inference can run on an $8 microcontroller showcases how memory-mapping innovation can bypass severe hardware memory limits for edge AI.
- –Efficient Memory Mapping: Storing the 25M parameter embedding matrix in SPI flash memory allows the chip to stream token rows on demand without needing heavy SRAM or external PSRAM allocations.
- –Respectable Edge Throughput: Generating ~9.5 tokens per second delivers real-time performance suitable for low-cost standalone devices and interactive hardware toys.
- –Proof-of-Concept for Micro-LLMs: While specialized for TinyStories generation, the approach establishes a clear blueprint for running targeted micro-models on consumer microcontrollers completely offline.
// TAGS
esp32-aiesp32edge-aitinystoriesembedded-systemsopen-sourcemicrocontrollersllmtiny-lm
DISCOVERED
1h ago
2026-07-27
PUBLISHED
1h ago
2026-07-27
RELEVANCE
8/ 10
AUTHOR
Github Awesome