Discriminative vs. Generative Models

Here is a fact worth sitting with. Almost every model this course has discussed so far, the spam classifiers, the price predictors, the fraud detectors, shares one thing in common. They are all discriminative models. Generative AI, the subject of this course's entire second phase, is a genuinely different kind of task, and this chapter is where that difference gets named precisely.

A discriminative model learns to distinguish between categories, or to predict a value, given an input. It draws a boundary or a mapping, and its whole job is telling you which side of that boundary a new example falls on, or what value it maps to. Is this email spam? What will this house sell for? Does this image contain a cat? Every one of these is a discriminative task: given an input, produce a judgment about it.

A generative model learns something different: not how to judge an existing example, but how to produce entirely new examples that plausibly could have come from the same underlying distribution as its training data. Instead of asking "is this a picture of a cat," a generative model learns what pictures of cats generally look like, well enough to produce a brand new image of a cat that never existed before. Instead of asking "is this sentence grammatical," a generative model learns what plausible language generally looks like, well enough to write an entirely new sentence.