GenAI Learn/Evals Foundations

Browsing as a guest. Sign in to save your progress and earn XP as you complete chapters.

Code-Based Graders & the Metrics That Mislead

6 min read

You'll learn to

-Use plain code to grade everything a rule can decide, including real outcomes
-Explain why overlap metrics like BLEU and ROUGE mislead on open-ended answers
-Spot the trap of generic scores such as "helpfulness 4.2"
-Choose between a code grader and a judge for a given check

Before you pay a model to judge anything, ask a cheaper question: can a few lines of code decide this? Code graders are fast, free to run on every change, and give the same answer every time. They are also easy to misuse, and a few famous metrics look like graders but measure the wrong thing. This chapter covers both sides.

An Analogy: The Spelling Checker and the Editor

A spelling checker is perfect for one job: finding words that are not in its word list. It is fast and never tired. But it cannot tell you whether a paragraph is persuasive. For that you need an editor. Code graders are the spelling checkers of AI quality. Use them for everything they can decide, and save the editors, the judges, for what they cannot.

What Code Can Grade

-Format: is the output valid JSON, does it match a schema, is it under the length limit?
-Required and forbidden content: does it mention the refund window, does it avoid leaking an internal ID or an email address?
-Tool use: did the agent call the right tool with the right arguments, in a sensible order?
-Correctness with a checkable answer: run the generated code against unit tests, compare a computed number, query the database.
-Outcome: is the world in the state it should be in afterward?

Grade the Outcome, Not the Story

An agent's transcript can say anything. The database says what happened. Anthropic's guidance on agent evals makes the same point: grade what the agent produced, not the exact path it took, because paths are brittle and a clever valid route should not be punished. The next block runs two trials of a refund agent. Both end with the agent saying it is done. Press Run and see which grader catches the difference.

Press Run: the transcript is the same in both trials, the outcome is not

# A tiny stand-in for the real world: the refunds table after each trial.
def run_trial(name, agent_says, refund_rows):
    return {"name": name, "transcript_final": agent_says, "refunds_table": refund_rows}

SAID = "Done! I have issued your refund of $40."
trials = [
    run_trial("trial 1", SAID, [{"order": 48213, "amount": 40}]),
    run_trial("trial 2", SAID, []),          # claimed, but nothing was written
]

def claim_grader(t):
    return "issued your refund" in t["transcript_final"].lower()

def outcome_grader(t, order=48213, amount=40):
    return any(r["order"] == order and r["amount"] == amount for r in t["refunds_table"])

for t in trials:
    print(f"{t['name']}: claim says {'pass' if claim_grader(t) else 'fail'}, "
          f"outcome says {'pass' if outcome_grader(t) else 'FAIL'}")

The claim grader passes both trials. Only the outcome grader notices that trial 2 never wrote the refund. In a real harness the "refunds table" is your database or a test double of it, started from a clean state for each trial so one trial cannot affect the next.

The Metrics That Mislead

For years, text generation was scored by comparing the output to a reference answer with an overlap formula. BLEU, built for machine translation, measures how many word groups match. ROUGE, built for summaries, measures how much of the reference the output recovers. Both reward matching words, and that is the problem. Eugene Yan's survey of LLM patterns notes the poor correlation between these metrics and human judgment, and that scores were hard to reproduce across implementations.

A related metric, BERTScore, compares meaning through embeddings, so it handles synonyms and paraphrase better. It is a reasonable tool for search and retrieval debugging. It is not a substitute for deciding whether an answer is correct.

The next block makes the failure concrete. The reference says returns are accepted within 30 days. One candidate answer says 3 days, which is wrong and shares most of its words. Another says "you have a month", which is right and shares almost none. Press Run to see how a simple overlap score and a simple code grader treat each one.

Press Run: the overlap score ranks the wrong answer above the right paraphrase

import re

def tokens(text):
    return re.findall(r"[a-z0-9]+", text.lower())

def overlap_f1(reference, candidate):
    """A simple unigram overlap score, the core idea behind ROUGE-1."""
    ref, cand = tokens(reference), tokens(candidate)
    common = 0
    ref_left = list(ref)
    for w in cand:
        if w in ref_left:
            ref_left.remove(w)
            common += 1
    if common == 0:
        return 0.0
    precision, recall = common / len(cand), common / len(ref)
    return 2 * precision * recall / (precision + recall)

def days_grader(answer, expected=30):
    """A code grader: pull out a number of days, accept 'a month' for 30."""
    text = answer.lower()
    m = re.search(r"(\d+)\s*days?", text)
    if m:
        return int(m.group(1)) == expected
    if "month" in text:
        return expected == 30
    return False

REFERENCE = "Returns are accepted within 30 days of delivery."
CANDIDATES = [
    ("wrong number, similar words", "Returns are accepted within 3 days of delivery.", False),
    ("right answer, new words",     "You have a month after it arrives to send it back.", True),
    ("right answer, same words",    "Returns are accepted within 30 days of delivery.", True),
]

print(f"{'candidate':<30}{'overlap':>9}{'code grader':>14}{'truth':>9}")
for label, answer, truth in CANDIDATES:
    score = overlap_f1(REFERENCE, answer)
    verdict = days_grader(answer)
    verdict_text = "pass" if verdict else "fail"
    truth_text = "right" if truth else "wrong"
    print(f"{label:<30}{score:>9.2f}{verdict_text:>14}{truth_text:>9}")

import re

def tokens(text):
    return re.findall(r"[a-z0-9]+", text.lower())

def overlap_f1(reference, candidate):
    """A simple unigram overlap score, the core idea behind ROUGE-1."""
    ref, cand = tokens(reference), tokens(candidate)
    common = 0
    ref_left = list(ref)
    for w in cand:
        if w in ref_left:
            ref_left.remove(w)
            common += 1
    if common == 0:
        return 0.0
    precision, recall = common / len(cand), common / len(ref)
    return 2 * precision * recall / (precision + recall)

def days_grader(answer, expected=30):
    """A code grader: pull out a number of days, accept 'a month' for 30."""
    text = answer.lower()
    m = re.search(r"(\d+)\s*days?", text)
    if m:
        return int(m.group(1)) == expected
    if "month" in text:
        return expected == 30
    return False

REFERENCE = "Returns are accepted within 30 days of delivery."
CANDIDATES = [
    ("wrong number, similar words", "Returns are accepted within 3 days of delivery.", False),
    ("right answer, new words",     "You have a month after it arrives to send it back.", True),
    ("right answer, same words",    "Returns are accepted within 30 days of delivery.", True),
]

print(f"{'candidate':<30}{'overlap':>9}{'code grader':>14}{'truth':>9}")
for label, answer, truth in CANDIDATES:
    score = overlap_f1(REFERENCE, answer)
    verdict = days_grader(answer)
    verdict_text = "pass" if verdict else "fail"
    truth_text = "right" if truth else "wrong"
    print(f"{label:<30}{score:>9.2f}{verdict_text:>14}{truth_text:>9}")

The overlap score gives the wrong answer 0.88 and the correct paraphrase 0.00, the exact opposite of the truth. The code grader gets all three right, but notice why: it was written for this one fact, with a rule for "a month". That is the trade-off. Code graders are precise where you can write the rule, and silent about everything you did not think of. When the space of correct wordings is too wide for rules, you need a judge, which is the subject of the next module.

The Generic-Score Trap

Many tools ship ready-made scores called helpfulness, coherence, toxicity, or hallucination, usually from a model. They are tempting because they produce a tidy chart. The risk is false confidence. A feature can score 4.2 out of 5 on helpfulness while failing the one thing your product exists to do. Practitioners who teach evals are blunt about it: use generic scores, at most, to find traces worth reading, never as proof of quality. Measure the specific failures you found during error analysis.

Choosing Code or a Judge

A quick rule of thumb

-Use code when: The answer is checkable: a format, a number, a required phrase, a test result, a database state.
-Use a judge when: Correctness depends on meaning or tone and many wordings are valid: summaries, explanations, empathy, faithfulness to a source.
-Use a person when: Stakes are high, the criteria are still being discovered, or you are checking the judge itself.

Always try a code check first, even for fuzzy-sounding problems. Often a failure that seems to need a judge, like "the bot leaked an internal ID", is really a pattern a few lines of code can catch.

Interview Signal is part of Pro

See a real weak answer next to a real strong one for this exact topic.

Quiz is part of Pro

Test what you just read with a short quiz, and bank the XP.

Building Eval Datasets LLM-as-Judge, Done Right

Discussion0

Join the Discussion

No comments yet. Be the first to start the thread!