OPEN_SOURCE ↗
REDDIT · REDDIT// 17d agoPOLICY REGULATION
ICML 2026 review policies may skew scores
A Reddit post from ICML 2026 asks whether papers reviewed under no-LLM Policy A are getting harsher scores than papers under permissive Policy B. The author says the signal is anecdotal and is collecting an informal poll to compare score vectors and review style across batches.
// ANALYSIS
The signal may be real, but the cleaner explanation is probably a mix of reviewer-composition effects, author self-selection, and LLM-polished tone rather than a pure policy-driven scoring bias.
- –Policy B still forbids LLMs from judging strengths/weaknesses or writing the review, so any score gap would likely come from tone/compliance differences rather than an official scoring shortcut.
- –Policy A papers are guaranteed Actual Policy A reviewers, while Policy B papers draw a mixed A/B pool, so raw score comparisons are confounded before any LLM effect shows up.
- –The Reddit/X sample is self-selected, and people with strong feelings are more likely to respond, so the poll is better for hypothesis-building than for proof.
- –ICML says it will analyze score distributions and compensate systematic differences, which means this is a live process-design issue, not just forum gossip.
// TAGS
icml-2026llmresearchregulationethics
DISCOVERED
17d ago
2026-03-25
PUBLISHED
18d ago
2026-03-25
RELEVANCE
7/ 10
AUTHOR
Available_Net_6429