OPEN LINK ↗
// 45d agoPRODUCT UPDATE
Krasis 1.0 ships Rust, HQQ, Qwen support
Krasis hits v1.0 with a pure Rust/CUDA rewrite, enabling high-speed inference of 35B+ models on consumer GPUs. The update adds sensitivity-aware HQQ attention and 4-bit KV caching to run massive models on hardware as low as an 8GB laptop GPU.
// ANALYSIS
Krasis is solving the "VRAM wall" for developers by making massive models like Qwen 3.6 usable on consumer-grade hardware.
- –Moving to a pure Rust hot path eliminates GIL bottlenecks, resulting in dramatic performance gains for both prefill and decode
- –Sensitivity-aware HQQ uses mixed precision to keep memory usage low while moving sensitive model areas to higher precision
- –New 4-bit and 6-bit KV cache implementations are fully Ampere-compatible, avoiding the need for FP8 hardware
- –Hybrid Cold Swap system dynamically manages VRAM pressure, allowing 122B parameter models to run on single-GPU setups
- –The HQQ assessment doesn't require pre-built templates, making the runtime significantly more flexible for new model support
// TAGS
krasisllmquantizationinferenceopen-sourceopen-weightsgpulocal-first
DISCOVERED
45d ago
2026-05-28
PUBLISHED
45d ago
2026-05-28
RELEVANCE
8/ 10
AUTHOR
mrstoatey