Probability & Uncertainty for ML

Machine learning models almost never output a flat yes or no. They output probabilities, degrees of confidence, and it is up to the surrounding system to decide what to do with that number. Get comfortable with this idea now, because it shapes everything else in this course.

When a spam classifier says an email is 92% likely to be spam, it is not stating a fact. It is reporting a degree of belief based on patterns in its training data. That remaining 8% chance of being wrong is not a bug. It is an honest reflection of genuine uncertainty. A model that always claimed 100% or 0% confidence would usually be lying about how sure it actually is.

Somewhere in the system, a threshold decides what probability counts as "spam enough" to act on: move to a spam folder at 90% confidence, flag for review between 50% and 90%, deliver normally below that. Choosing that threshold is an engineering and product decision, not something the model decides for you, and it directly trades off false positives against false negatives.

The single most interview relevant piece of probability in ML is Bayes' theorem: a formula for updating a prior belief once new evidence arrives. It is the mathematical backbone of spam filters, medical test interpretation, and, conceptually, how a model's confidence should shift as it sees more signal.

Here is a genuinely counterintuitive example. A medical test is 99% accurate for a disease that only 1 in 10,000 people actually have. You test positive. How worried should you be? Intuition says very, but Bayes' theorem disagrees, because the disease is so rare that false positives from the healthy 9,999 vastly outnumber true positives from the one person who is actually sick.