1-bit Bonsai 8B runs on legacy 2GB GPU

The successful execution of an 8B parameter model on a 2GB card is a watershed moment for architectural efficiency, proving that native 1-bit training can bypass the hardware floor previously required for usable AI.

• –Native 1-bit weights reduce the model footprint to just 1.15GB, finally enabling 8B-class reasoning on devices previously restricted to tiny SLMs.
• –Thermal constraints remain the primary bottleneck for legacy mobile GPUs, with the MX150 quickly hitting 80°C and losing 30-40% of its performance.
• –Memory management is a critical trade-off, as fitting the model on a 2GB card requires aggressive KV cache quantization (q8_0) and limits context to ~5.6k tokens.
• –At 6 Joules per token, the energy efficiency on older 16nm/14nm silicon is poor compared to modern NPUs, making this a feat of "possibility" rather than a recommendation for production use.
• –The use of custom 1-bit kernels in a specialized llama.cpp fork underscores the need for new software standards to support non-standard bit-depths.