QoRL beats Postgres query optimizer by 81%

Relational query optimization has been an intractable heuristic bottleneck for decades, and this experiment demonstrates that small, domain-specialized language models running agentic RL against verifiable runtime feedback can systematically outmaneuver hand-tuned database engines at minimal compute cost.

• –**Verifiable execution feedback makes systems optimization an ideal RL domain:** Unlike conversational benchmarks plagued by reward hacking and noisy LLM judges, query latency provides a deterministic, scalar reward signal that allows agentic post-training to converge reliably.
• –**Asynchronous worker leasing prevents inference bottlenecks:** Recognizing that 92% of rollout duration was consumed by LLM token generation rather than database execution, the architecture decoupled vLLM inference on remote GPUs from local PostgreSQL measurement containers, scaling concurrency from 4 to 20 parallel rollouts.
• –**Guiding existing optimizers via hints beats end-to-end replacement:** Rather than attempting to rewrite Postgres from scratch, the system uses pg_hint_plan to steer join trees, join algorithms, and scan methods, pairing modern deep learning exploration with proven relational execution engines.
• –**High-impact specialization on consumer-grade compute:** The entire training workflow cost roughly $1,200 in rented compute and API demonstrations, illustrating that enterprises can adapt small open models to optimize proprietary, mission-critical infrastructure without frontier-scale budgets.