YOU ARE VIEWING ONE ITEM FROM THE AICRIER FEED

Alconost drops pro-grade MQM gold dataset

AICrier tracks AI developer news across Product Hunt, GitHub, Hacker News, YouTube, X, arXiv, and more. This page keeps the article you opened front and center while giving you a path into the live feed.

// WHAT AICRIER DOES

7+

TRACKED FEEDS

24/7

SCRAPED FEED

Short summaries, external links, screenshots, relevance scoring, tags, and featured picks for AI builders.

Alconost drops pro-grade MQM gold dataset
OPEN LINK ↗
// 71d agoOPENSOURCE RELEASE

Alconost drops pro-grade MQM gold dataset

Alconost has open-sourced its MQM-annotated MT evaluation dataset on Hugging Face (https://huggingface.co/datasets/alconost/mqm-translation-gold), with 362 segments across 16 language pairs and annotations from 48 professional linguists. The accompanying announcement (https://www.reddit.com/r/MachineLearning/comments/1rw3a3j/d_releasing_a_professional_mqmannotated_mt/) positions it as a WMT-aligned, higher-agreement alternative to noisier crowdsourced test sets.

// ANALYSIS

This is the kind of small-but-clean dataset release that can punch above its size in MT eval workflows.

  • The data includes full MQM structure (category, severity, span) plus multiple annotators per segment, which is unusually useful for agreement and metric-analysis work.
  • Reported Kendall’s τ = 0.317 is materially above typical WMT ranges cited by the authors, suggesting annotation process quality was a priority.
  • The dataset is better suited for benchmarking, error analysis, and reward-model calibration than for training large translation models due to its scale.
  • License is CC BY-SA 4.0 and the card notes it is a growing collection, so it could become a recurring reference set if updates continue.
  • If replicated by others, this could raise expectations for professionally annotated open MT eval resources.
// TAGS
mqm-translation-goldmachine-translationllmbenchmarkresearchopen-sourcehuman-evaluation

DISCOVERED

71d ago

2026-03-17

PUBLISHED

71d ago

2026-03-17

RELEVANCE

8/ 10

AUTHOR

ritis88