PrismML Ternary Bonsai shows split speeds

The hot take: this is a backend story more than a model story. PrismML's weights may be interesting, but the real differentiator right now is whether you are on the well-tuned MLX path or a still-maturing llama.cpp/CUDA stack.

• –The Mac MLX numbers are genuinely strong for local inference, especially the 8B model at roughly 41 t/s on a 4K context.
• –The Windows CPU-only results are weak enough that TTFT becomes the headline problem, not just steady-state tokens/sec.
• –PrismML's own docs say ternary Q2_0 inference is currently in its llama.cpp fork, with CUDA support tied to that path, so user results depend heavily on backend readiness.
• –The comparison to the 1-bit Bonsai CPU build suggests the ternary format may be trading compatibility and kernel maturity for the compression headline.
• –For developers evaluating these models, apples-to-apples backend comparisons matter more than raw model claims; the same model family can look excellent or disappointing depending on the runtime.