AgentSeal audits contamination behind eval scores

Benchmark scores become recall tests when models can access benchmark questions, answers, or close replicas; AgentSeal’s strongest contribution is separating those exposure signals instead of collapsing them into an overconfident “contaminated” verdict.

• –Audits SWE-bench-style coding benchmarks for patch, test, problem-statement, and repository exposure
• –Combines deterministic content overlap with probabilistic corpus-membership checks and public-source evidence
• –Reports evidence classes separately, making findings easier to triage and reproduce
• –Explicitly warns that public availability does not prove a model memorized the material
• –Reinforces the need for fresh, sealed, independently authored eval sets alongside contamination audits