Multiscreen architecture replaces softmax with absolute relevance screening

Softmax's relative weight distribution has been the fundamental bottleneck for sparsity and long-context scaling; Multiscreen's thresholding finally solves the "always attending" problem.

• –40% fewer parameters with comparable validation loss to traditional Transformers.
• –3.2x inference speedup at 100k context length via sparse computation skipping.
• –Enables stable training at significantly higher learning rates (up to 0.0625).
• –Spartan attention: models learn to attend only to relevant keys, often screening out 95% of the context.