Nick Lothian SQL benchmark crowns Qwen 122B

The big takeaway is that tool-calling quality now matters almost as much as raw model size for SQL agents, and a well-tuned small model can close a lot of the gap. But this is still a narrow benchmark: it’s a strong read on single-shot agentic SQL, not a proxy for broader codebase reasoning.

• –Qwen3.5-122B-A10B is the clear heavyweight winner here, with RAM offload making it usable on 16GB VRAM cards if you can tolerate slower inference
• –Qwen3.5-9B Claude-4.6 HighIQ is the practical surprise: most of its earlier failures came from malformed tool calls, so the finetune is doing real work, not just posturing
• –Nemotron-Cascade-2-30B-A3B looks unusually competitive for its size and deserves attention as a self-hostable sweet spot
• –The benchmark is deliberately short and agentic, so models that are good at isolated SQL generation can shine even if they may not generalize to longer multi-step coding tasks
• –For local LLM users, this reinforces the tradeoff triangle: bigger models still win on quality, but quantization, offload, and tool-call reliability decide what is actually usable