BACK_TO_FEEDAICRIER_2
Alconost drops pro-grade MQM gold dataset
OPEN_SOURCE ↗
REDDIT · REDDIT// 25d agoOPENSOURCE RELEASE

Alconost drops pro-grade MQM gold dataset

Alconost has open-sourced its MQM-annotated MT evaluation dataset on Hugging Face (https://huggingface.co/datasets/alconost/mqm-translation-gold), with 362 segments across 16 language pairs and annotations from 48 professional linguists. The accompanying announcement (https://www.reddit.com/r/MachineLearning/comments/1rw3a3j/d_releasing_a_professional_mqmannotated_mt/) positions it as a WMT-aligned, higher-agreement alternative to noisier crowdsourced test sets.

// ANALYSIS

This is the kind of small-but-clean dataset release that can punch above its size in MT eval workflows.

  • The data includes full MQM structure (category, severity, span) plus multiple annotators per segment, which is unusually useful for agreement and metric-analysis work.
  • Reported Kendall’s τ = 0.317 is materially above typical WMT ranges cited by the authors, suggesting annotation process quality was a priority.
  • The dataset is better suited for benchmarking, error analysis, and reward-model calibration than for training large translation models due to its scale.
  • License is CC BY-SA 4.0 and the card notes it is a growing collection, so it could become a recurring reference set if updates continue.
  • If replicated by others, this could raise expectations for professionally annotated open MT eval resources.
// TAGS
mqm-translation-goldmachine-translationllmbenchmarkresearchopen-sourcehuman-evaluation

DISCOVERED

25d ago

2026-03-17

PUBLISHED

26d ago

2026-03-17

RELEVANCE

8/ 10

AUTHOR

ritis88