Decision Models Make LLM Routing One-Pass

This is a compelling pattern for routing and triage, but constrained outputs only guarantee valid choices—not correct ones.

• –Single-pass inference can cut latency and cost for repetitive classification tasks
• –Raw token probabilities are badly overconfident without calibration
• –Temperature scaling improves confidence estimates, but requires representative evaluation data
• –Fixed option sets work well for routing, moderation, and scoring, not open-ended reasoning
• –The pattern complements generative LLMs: use decision models for branching and larger models for synthesis