PinPoint toughens multimodal search evals

PinPoint looks more useful as evaluation infrastructure than as another leaderboard paper because it targets the exact failure modes glossy retrieval demos usually hide.

• –Explicit negatives and multiple correct answers make this benchmark much closer to real search quality than recall-only setups where models can bury wrong results among a few right ones
• –The six paraphrases per query expose how brittle current systems still are to phrasing, with the paper reporting up to 25.1% performance variation across rewordings
• –Multi-image composition remains a major open problem: even strong methods reportedly lose 48-72% of their performance on multi-image queries
• –Releasing the full dataset, retrieval index, baseline results, and evaluation code gives smaller research groups a much better foundation for reproducible multimodal search work