OpenAI researcher explains why AI models hallucinate

This insight highlights a fundamental challenge in current AI training paradigms regarding alignment and reward functions. The training incentives inadvertently reward confidently incorrect answers over honest admissions of ignorance, emphasizing the distinction between a model genuinely knowing a fact versus merely recognizing the syntactic structure of a correct answer. Addressing hallucinations may require foundational shifts in how models are evaluated during training rather than relying solely on post-training filters.