OPEN LINK ↗
// 71d agoBENCHMARK RESULT
Mistral Small 4 vision draws early backlash
A LocalLLaMA discussion reports that Mistral Small 4 misreads a straightforward concert photo even through Mistral’s official API, hallucinating stadium elements that are not present. The post contrasts this with much stronger outputs from smaller competitors and notes older Mistral small models did not show the same failure pattern.
// ANALYSIS
The hot take is that Mistral Small 4 may have shipped with a meaningful real-world vision reliability gap despite strong “unified multimodal” positioning.
- –The failure mode is not subtle: the model invents core scene structure (stadium, track, vehicles), which breaks trust for visual workflows.
- –Because the author reproduced the issue on the official API, the thread shifts blame away from local quantization/runtime setup.
- –Community comparisons to Qwen and prior Mistral small variants suggest a possible regression rather than normal variance.
- –For developers, this looks like a “benchmark vs. product reality” warning: run task-specific image evals before adopting Small 4 in production.
// TAGS
mistral-small-4mistralllmmultimodalbenchmarkapiopen-weights
DISCOVERED
71d ago
2026-03-17
PUBLISHED
71d ago
2026-03-17
RELEVANCE
8/ 10
AUTHOR
EffectiveCeilingFan