The Perceptron: A Single Neuron

A neural network, no matter how large, is built out of a huge number of copies of one very simple unit: the artificial neuron, originally called the perceptron. Understand this one small piece thoroughly and the rest of the module falls into place quickly.

A neuron takes several numeric inputs, multiplies each one by its own learned weight, adds them all up along with one extra learned number called a bias, and passes that sum through a function that produces the neuron's output. That is the entire mechanism, no more and no less. If this sounds familiar, it should. It is mathematically the same weighted-sum operation behind linear and logistic regression from the Classical ML Toolbox module.

The weights and bias are exactly the numbers that training adjusts. The original perceptron learning rule is refreshingly simple. For each training point the perceptron currently gets wrong, nudge the weights a little in the direction that would have made it right. Watch that process happen step by step below, starting from a deliberately bad initial guess, as the boundary rotates and shifts until every point is correctly classified.

The original 1958 Perceptron was exactly this single unit, and it turned out to have a hard mathematical limit. A single neuron can only separate data with a straight line, or a flat plane in higher dimensions. It cannot represent problems where the correct boundary is a curve or a more complex shape, like the classic XOR pattern, where the two classes form an X that no single straight line can split. That limitation nearly ended the field in the late 1960s, and it is exactly what the next chapter's idea, stacking neurons into layers, was invented to overcome.