Support tokens boost LLM robustness

This research creates a rigorous bridge between modern transformers and classical statistical learning theory (SVMs), offering a much-needed geometric interpretation of attention.

• –The "support tokens" concept provides a new, intuitive lens for interpretability: identifying which sequence positions are closest to a "degeneracy boundary."
• –The MAP-style training penalty is a practical, low-overhead win, improving robustness with minimal loss in clean accuracy.
• –Reframing self-attention as a change-of-variables constraint reveals a stability margin that can be directly optimized during training.
• –The probabilistic framing suggests that LLMs can be modeled as a stochastic process over the power set of the token space.
• –This represents a move towards more grounded, theoretically-sound architectures rather than purely empirical "black box" scaling.