Human review finds errors in clean TranslateGemma subtitles

This follow-up benchmark argues that TranslateGemma’s strong automatic scores may overstate real subtitle quality in certain cases. The authors re-checked 84 translation segments that both MetricX-24 and COMETKiwi had marked as clean, then had professional linguists annotate them with MQM. Even in this high-confidence zone, most segments were flagged by humans, including many accuracy errors that the metrics completely missed. The post is careful about scope, but it raises a real question about whether reference-free QE metrics are too forgiving for subtitle translation.