Small LLMs reveal primitive semantic layer

Preliminary and self-published, but the cross-architecture consistency is hard to dismiss — four different model families showing the same structural distinction demands at least a second look.

• –The Layer 0a/0b split maps loosely onto linguistic notions of function vs. content words; if real at the activation level, it implies LLMs encode semantic structure rather than pure distributional statistics
• –The inverse scaling pattern — gap largest in 360M models, narrowest in 1B — is the most provocative finding: larger models may develop phenomenological access to scaffolding primitives, which could partially explain emergent capability thresholds
• –11 validated two-primitive compositions (WANT + GRIEF → longing, FEEL + GRIEF → heartbreak) suggest compositionality in the primitive layer, not just isolated activation differences
• –Acknowledged circularity: the classifier measuring activation is the same class of model being measured — a real methodological concern the authors flag openly
• –Fully reproducible locally via Ollama with no API keys — low barrier to independent verification