llamAmpere v0.6.3 Runs 125B MoE Locally

This is a striking demonstration of how model-specific kernels, quantization, speculative decoding, and GPU/RAM expert placement can make a 125B MoE practical on aging consumer hardware—but the numbers are creator-reported and configuration-sensitive.

• –Reported performance reaches 120+ tok/s generation at 100K-token context depth, with prompt processing above 3,000 tok/s. [Benchmark details](https://www.reddit.com/r/LocalLLaMA/comments/1xpmzg7/50_lower_error_than_strata_3xs_27_lower_error/)
• –The new 80GB AI3 model file keeps more routed experts on the GPU while using system RAM for the remainder.
• –The project claims lower probability error than competing Strata and Unsloth quantizations, though KLD is not equivalent to task accuracy.
• –Hardware requirements remain significant: a 24GB RTX 3090 setup needs ample system RAM, while 12GB configurations require roughly 64GB of system memory.
• –The broader takeaway is that specialized inference stacks are narrowing the gap between local experimentation and hosted-model responsiveness.