OPEN LINK ↗
// 1h agoMODEL RELEASE
Qwen3.8-Flash-Next Previews Qwen4 Architecture
Qwen has open-sourced a multimodal MoE model designed as an early preview of Qwen4’s architecture. Its 125B-parameter core activates just 6B parameters per token, supports 262K-token context natively, and uses sparse attention to reduce long-context inference costs.
// ANALYSIS
Qwen is making architectural efficiency the headline, not merely scaling parameter counts—a compelling direction for developers building long-context and agentic systems.
- –Qwen Sparse Attention selects relevant context at micro-block granularity, reducing both attention computation and indexing overhead
- –N-gram embeddings add 51B parameters that can be offloaded to host memory, expanding capacity without consuming equivalent GPU memory
- –The model combines Gated DeltaNet, sparse attention, gated residuals, and ultra-sparse MoE routing in a distinctly hardware-conscious design
- –Open weights are already supported by Transformers, vLLM, SGLang, and other inference stacks, lowering experimentation friction
- –The production Qwen3.8-Flash version reportedly offers 1M-token context through Qwen Cloud at low API pricing, but real-world hardware requirements remain substantial
// TAGS
qwen3.8-flash-nextllmopen-weightsmultimodalmoelong-contextinference
DISCOVERED
1h ago
2026-08-26
PUBLISHED
2h ago
2026-08-26
RELEVANCE
10/ 10
AUTHOR
omarsar0