Novel LLM-26 Hunts LLM Blind Spots

This is less a demo of model failure than a demonstration of how shallow many “smart” answers still are: the model often nails the familiar puzzle shape before it actually counts. The repo is interesting because it automates adversarial discovery, which is closer to useful eval infrastructure than another one-off benchmark.

• –The workflow matters more than the individual riddle: it spins multiple independent agents, scores consensus, and keeps iterating until it finds a low-agreement question.
• –The “strawperrry” example is a clean reminder that long context and higher effort do not eliminate tokenization and pattern-matching errors.
• –The project sits in the useful middle ground between benchmark and agent harness, so it could be adapted into a broader eval pipeline for model QA.
• –The risk is overfitting to puzzle-style failures; these are good canaries, but they do not fully represent real-world reasoning robustness.