BACK_TO_FEEDAICRIER_2
ICML 2026 review policies may skew scores
OPEN_SOURCE ↗
REDDIT · REDDIT// 17d agoPOLICY REGULATION

ICML 2026 review policies may skew scores

A Reddit post from ICML 2026 asks whether papers reviewed under no-LLM Policy A are getting harsher scores than papers under permissive Policy B. The author says the signal is anecdotal and is collecting an informal poll to compare score vectors and review style across batches.

// ANALYSIS

The signal may be real, but the cleaner explanation is probably a mix of reviewer-composition effects, author self-selection, and LLM-polished tone rather than a pure policy-driven scoring bias.

  • Policy B still forbids LLMs from judging strengths/weaknesses or writing the review, so any score gap would likely come from tone/compliance differences rather than an official scoring shortcut.
  • Policy A papers are guaranteed Actual Policy A reviewers, while Policy B papers draw a mixed A/B pool, so raw score comparisons are confounded before any LLM effect shows up.
  • The Reddit/X sample is self-selected, and people with strong feelings are more likely to respond, so the poll is better for hypothesis-building than for proof.
  • ICML says it will analyze score distributions and compensate systematic differences, which means this is a live process-design issue, not just forum gossip.
// TAGS
icml-2026llmresearchregulationethics

DISCOVERED

17d ago

2026-03-25

PUBLISHED

18d ago

2026-03-25

RELEVANCE

7/ 10

AUTHOR

Available_Net_6429