OPEN LINK ↗
// 1h agoINFRASTRUCTURE
Furiosa SDK 2026.4 boosts RNGD inference
FuriosaAI’s latest SDK improves RNGD inference with profile-guided decode batching, faster sampling, hierarchical KV caching, optimized multimodal serving, and broader model support. The release reports up to 138% higher token-generation performance per watt across tested models.
// ANALYSIS
Furiosa is pushing RNGD beyond raw accelerator performance into a more complete serving stack, though the upgrade carries meaningful migration costs.
- –Profile-guided batching is now enabled by default, delivering a median 22.7% throughput gain on GPT-OSS-120B benchmarks.
- –Experimental HiCache extends prefix reuse from NPU memory into host memory and shared Mooncake storage.
- –Multimodal serving gains continuous batching, batched vision encoding, and reusable image embeddings.
- –Support expands across Gemma 4, EXAONE 4.5, Qwen3-VL, embedding, reranking, and FP8 Llama variants.
- –Operators must upgrade the RNGD driver and migrate legacy artifacts and build workflows to FXB.
// TAGS
furiosa-sdkinferencesdkmultimodalllmedge-ai
DISCOVERED
1h ago
2026-10-16
PUBLISHED
2h ago
2026-10-16
RELEVANCE
8/ 10
AUTHOR
FuriosaAI