Supervised Learning: Labeled Examples
You'll learn to
- -Define supervised learning and the role of labeled data
- -Distinguish classification from regression as the two main supervised tasks
- -Write the empirical risk minimization objective that every supervised algorithm is ultimately solving
- -Implement and run a mean-squared-error calculation from scratch
Supervised learning is the most common and most intuitive type of machine learning. You give the algorithm many examples of inputs paired with the correct output, called labels, and it learns a function that maps new, unseen inputs to the correct output as accurately as possible.
The Two Supervised Tasks
- -Classification predicts a category. Is this email spam or not? Is this image a cat, dog, or neither? The output is one of a fixed set of labels.
- -Regression predicts a number. What will this house sell for? How many minutes until this delivery arrives? The output is a continuous value.
Both tasks follow the same recipe. Collect labeled examples, called the training set. Let an algorithm adjust itself to fit those examples. Then evaluate how well it generalizes to examples it has never seen.
The One Formula Every Supervised Algorithm Is Solving
Underneath the specific names, linear regression, decision trees, neural networks, every supervised learning algorithm is minimizing the same abstract quantity: the average loss across the training set. This is called empirical risk minimization, and if you can hold this one formula in your head, you can explain what "training" means for any supervised model you will ever encounter.
- -θ (theta) refers to the model's parameters: the weights in a regression, the split thresholds in a tree, the millions of numbers in a neural network.
- -f(xᵢ; θ) is the model's prediction for input xᵢ, given its current parameters.
- -L(prediction, yᵢ) is the loss function comparing that prediction to the true label. The Math module covers loss functions in depth.
- -argmin_θ means search over every possible θ, and return the one that makes this quantity smallest.
This is worth memorizing for interviews. Whenever someone asks what a model is actually optimizing, the honest answer is always some version of this formula: a specific f, a specific L, and a specific optimization procedure, like gradient descent, for searching over θ.
The Catch: Labels Are Expensive
The word "supervised" refers to the labels acting as a supervisor, correcting the model's guesses during training. Getting those labels is often the hardest and most expensive part of the whole process. It usually means humans manually tagging thousands or millions of examples, which is why so much effort in the field goes into either reducing how many labels are needed, or finding ways to generate labels automatically.
A spam filter, a fraud detector, a medical image diagnosis tool, and a house price estimator are all supervised learning under the hood. Only the type of label, category or number, and the domain differ.
A product manager asks you to build a system that flags fraudulent transactions. What would you need before you could even start training a supervised model, and where would the real cost likely be?
"Just the transaction data, since a large enough model would figure out the pattern on its own."
"I would need not just raw transaction data but a large set of examples already labeled fraud or not fraud, since supervised learning requires known correct outputs to train against. In practice the real cost is usually not the model or the algorithm but the labels themselves, since accurately identifying which historical transactions were actually fraudulent tends to require human review or waiting for confirmed chargebacks. I would push early to understand exactly how those labels would be sourced and how reliable they are, since that determines whether the whole approach is even feasible."
What defines supervised learning?