Gemma 4, Qwen 3.6 chase harder vision tests

This is less a launch story than a benchmark-environment story, and that makes it more interesting for practitioners: the hard part in vision evals is not getting obvious demos right, it’s exposing where models break under ambiguity, clutter, and temporal noise.

• –The author already covered a strong baseline set: messy OCR, shelf OCR, geoguessing, meme understanding, table extraction, counting, sports tracking, fitness form checks, and AI-vs-real classification.
• –The best suggestions in the thread push into failure modes that usually separate models: scientific graphs, low-light wildlife cams, odd-angle edge detection, and noisy multi-object scans.
• –A useful comparison here depends on controlling preprocessing and token budgets; one commenter explicitly notes that Gemma’s image token settings materially affect results.
• –The post highlights a key gap in multimodal evals: models may describe images well yet still fail at localization, counting, measurement, or temporal consistency.
• –For local side-by-side testing, the most valuable prompts will be domain-specific and adversarial rather than generic benchmark samples.