Validating Your Judge: TPR, TNR & Human Agreement

A judge that has not been checked is a rumor. It produces confident verdicts, but you do not know how often they are right. Validation is how a judge earns the right to make decisions for you, and it takes one afternoon of labeling, not a research project.

A metal detector has two jobs. It must catch real threats, and it must not stop every traveler who has a belt buckle. Suppose only one passenger in a thousand carries something dangerous. A detector that never beeps is 99.9% accurate, because almost everyone is harmless, and it is completely useless. To judge a detector you need two numbers: how many real threats it catches, and how many harmless travelers it lets through. An LLM judge needs the same two numbers.

Pick one failure mode, such as "invents a policy". Have a domain expert label real outputs as pass or fail, with a one-line reason for each. Practitioners suggest 100 to 200 labeled examples per failure mode. Draw them from the traces you read during error analysis, and make sure the set has plenty of real failures. If only 1 output in 20 fails, a random sample of 100 gives you five failures, which is not enough to learn anything.

Treat a failure as the thing you want to catch. Compare the judge's verdicts with the human labels.

• -**True positive rate (TPR)**: of the outputs that really failed, the share the judge flagged. Eight of ten caught is 80%.
• -**True negative rate (TNR)**: of the outputs that really passed, the share the judge let through. 95 of 100 let through is 95%.
• -A low TPR means failures slip past. A low TNR means the judge cries wolf, and your team wastes time on false alarms.

Some teams define the positive outcome the other way around. Whichever way you choose, write down which is which, because a mixed-up definition produces confidently wrong conclusions.

"The judge agrees with humans 90% of the time" sounds strong. When failures are rare, it can mean nothing. The code below scores two judges on the same 100 labeled examples, of which only 10 are real failures. Press Run, and look at the agreement figure next to the TPR for the lazy judge that passes everything. Then read the last part, which uses the decent judge's two numbers to correct an estimate from production.

The lazy judge scores 90% agreement and catches 0% of failures. The decent judge scores 92% agreement, a number that barely differs, but it catches 80% of failures and lets 93% of good outputs through. That difference is invisible in the agreement figure and obvious in the two rates, which is why you report the two rates.

A judge also lets you estimate how often a failure happens in production, but its mistakes distort the count. Some good outputs get flagged, and some failures get missed. If the judge flags 15% of a sample, the real rate is not 15%. The last block above shows the correction. It solves one equation: the share flagged equals the failures the judge catches plus the good outputs it wrongly flags.

Here the correction lowers the estimate from 15.0% to about 11.4%, because the judge's false alarms were inflating it. The answer is still approximate. Only 10 failures sat behind that 80% catch rate, so the real catch rate could be quite different. That is exactly why the guidance asks for 30 to 50 failures in your dev and test sets, and why the next chapter is about noise.

Use the demo below with the numbers from this example: a judge that caught 8 of 10 failures and wrongly flagged 6 of 90 good outputs. Change the counts to see how the corrected rate moves, and how much it depends on only ten failures.

Treat your labeled examples the way you would treat data for a classifier. Otherwise you will tune the judge until it looks perfect on the examples it has already seen.

• -Train (roughly 10 to 20%): the source of the few-shot examples you paste into the judge prompt.
• -Dev (roughly 40 to 45%): where you test prompt changes and read the disagreements.
• -Test (roughly 40 to 45%): touched once, at the end, to report the final TPR and TNR.
• -Aim for 30 to 50 passes and 30 to 50 failures in both dev and test, so each rate rests on enough examples.

If the dev scores are much better than the test scores, you have overfit the prompt to the dev set. Go back, simplify the prompt, and collect more examples.

• -Run the judge on the dev set and list every disagreement with the human label.
• -Read them. Usually the definition is unclear, or the human label was debatable. Fix the definition in plain words before adding more rules.
• -Re-run, re-measure TPR and TNR, and stop when the numbers are good enough for the decision at hand.
• -Run once on the test set and record the result with the judge's prompt and model version.
• -Re-validate when the judge model changes, when the product changes, or when real traffic starts to look different.