12B models hit hallucination wall at 5k tokens

The 12B category is the local roleplay sweet spot, but architectural limits or quantization "toxic slop" are creating a hard ceiling for long-form narrative.

• –Performance degradation is model-agnostic across NemoMix, Rocinante, and Magnum, suggesting a shared root in the 12B base or common fine-tuning recipes
• –High temperatures (0.8+) accelerate the collapse, while lower settings only delay the inevitable fixation on specific descriptive patterns
• –"Context poisoning" means once the slop starts, switching models is futile as the new model inherits the broken linguistic patterns
• –DRY (Don't Repeat Yourself) samplers are becoming essential mitigations, yet they address symptoms rather than the underlying context-handling failure
• –Users are forced into a "treadmill" of retries at the 5k mark, highlighting a gap between marketed context windows and functional coherence