OPEN_SOURCE ↗
REDDIT · REDDIT// 25d agoOPENSOURCE RELEASE
Alconost drops pro-grade MQM gold dataset
Alconost has open-sourced its MQM-annotated MT evaluation dataset on Hugging Face (https://huggingface.co/datasets/alconost/mqm-translation-gold), with 362 segments across 16 language pairs and annotations from 48 professional linguists. The accompanying announcement (https://www.reddit.com/r/MachineLearning/comments/1rw3a3j/d_releasing_a_professional_mqmannotated_mt/) positions it as a WMT-aligned, higher-agreement alternative to noisier crowdsourced test sets.
// ANALYSIS
This is the kind of small-but-clean dataset release that can punch above its size in MT eval workflows.
- –The data includes full MQM structure (category, severity, span) plus multiple annotators per segment, which is unusually useful for agreement and metric-analysis work.
- –Reported Kendall’s τ = 0.317 is materially above typical WMT ranges cited by the authors, suggesting annotation process quality was a priority.
- –The dataset is better suited for benchmarking, error analysis, and reward-model calibration than for training large translation models due to its scale.
- –License is CC BY-SA 4.0 and the card notes it is a growing collection, so it could become a recurring reference set if updates continue.
- –If replicated by others, this could raise expectations for professionally annotated open MT eval resources.
// TAGS
mqm-translation-goldmachine-translationllmbenchmarkresearchopen-sourcehuman-evaluation
DISCOVERED
25d ago
2026-03-17
PUBLISHED
26d ago
2026-03-17
RELEVANCE
8/ 10
AUTHOR
ritis88