OPEN LINK ↗
// 1h agoOPENSOURCE RELEASE
1331 Labs Publishes Legal AI Errata Register
1331 Labs’ public register documents 16 objective corrections across eight legal AI repositories from six publishers, covering benchmarks, legal skills, plugins, and contract templates. Each entry provides fixed-commit evidence, authoritative sources, proposed corrections, and upstream tracking.
// ANALYSIS
The project highlights a neglected failure point in legal AI: flawed benchmarks and reference artifacts can make capable systems look wrong—or reward incorrect answers.
- –Corrections target Harvey, LegalQuants, Anthropic, General Legal, Thomson Reuters, and Crosby Legal artifacts.
- –Entries distinguish objective errors from subjective disagreements about benchmark design or drafting style.
- –Fixed commits, governing sources, arithmetic checks, and objection handling make the register independently auditable.
- –The repository warns that correction counts are not publisher quality rankings because artifact sizes and review depth vary.
- –For legal AI developers, benchmark and skill provenance deserves the same scrutiny as model selection.
// TAGS
legal-ai-errataevaluationbenchmarkopen-sourceresearchdataset
DISCOVERED
1h ago
2026-10-26
PUBLISHED
2h ago
2026-10-26
RELEVANCE
8/ 10
AUTHOR
SamRedlich