LLMs as Classifiers maps logprob uses

Logprobs are one of the few LLM outputs that can act like real engineering telemetry, but they are not a universal confidence metric. The signal is useful enough to build with, yet brittle enough that every team should validate it per model, prompt, and task.

• –Entropy is a reasonable way to surface ambiguous or mislabeled examples for data cleanup and active learning.
• –Log margin is a practical drift indicator when your input stream changes enough to shift the model’s confidence profile.
• –Threshold tuning with logprobs gives a native precision/recall control surface without bolting on a separate classifier.
• –The main limitation is calibration drift: absolute scores vary sharply across tasks and models, so cross-domain reuse is risky.
• –Best use case is operational triage, not ground truth; treat logprobs as a heuristic signal, not a guarantee.