Chunklaya unlocks long context for Laya

Non-autoregressive System 1 decision models represent an efficient, high-throughput alternative to generative LLMs for classification and routing, and chunklaya demonstrates that thoughtful passage-level orchestration can completely eliminate deep context blindness in small models without expensive fine-tuning.

• Overcoming Severe Decision-Head Bias: Laya's decision head fails to attend to information placed past token ~200; by chunking text into separate passages where each starts at position zero, chunklaya recovers needle-in-a-haystack detection AUC from chance to 1.000 on clean labels.

• Prompt Formulation as Calibration: Replacing raw boolean `noul` queries with 4-way descriptive `choice` questions dropped false-positive rates from 7.7% to 0.3% on identical weights, demonstrating how question structuring can compensate for uncalibrated model heads.

• Compute Parity on Consumer Hardware: Through length-sorted dynamic batching, chunklaya eliminates padding overhead and achieves roughly 483 ms per document on an M4 Pro chip, matching the execution speed of a single unchunked long forward pass.

• Pragmatic Local Alternative to Jev: Provides a fully offline, privacy-first option for developers wanting low-latency, deterministic decision-making without relying on closed cloud APIs like TypeSafe Jev.