Vertical vs Horizontal Scaling

Your single web server has been fine for months. Then traffic grows: the CPU sits near 95% at peak, pages slow down, and some requests time out. Something has to give, and you have exactly two directions to go: make that one machine more powerful, or add more machines next to it. These are vertical scaling (scale up) and horizontal scaling (scale out), and the choice between them shapes almost every architectural decision that follows.

A supermarket has one checkout lane and the queue is out of the door. There are two ways to fix it. You can make the lane faster: a quicker scanner, a longer belt, the best cashier in the chain. That is scaling up. Or you can open more lanes and put someone at the front to point each shopper at a free one. That is scaling out.

• -Making the one lane faster is vertical scaling: more CPU, more memory, faster disks in the same machine. Nothing else in the shop has to change.
• -Opening more lanes is horizontal scaling: more identical servers sharing the work. The person pointing shoppers at lanes is the load balancer, the topic of the next chapter.
• -Shoppers carry their own basket, so any lane can scan any basket. That is statelessness: no lane needs to remember anything about you, so lanes can be opened and closed freely.
• -A lane that breaks stops only its own queue. With a single lane, a breakdown closes the shop.

Vertical scaling means upgrading what you have: from 4 CPU cores to 64, from 8 GB of memory to 256 GB, from a spinning disk to a fast SSD. Its great virtue is simplicity, because your software does not change. If it ran on one server yesterday, it runs on one server today, only faster. For a quick win, or for a component that is genuinely hard to split (a database is the classic one), it is often the right first move.

It has three limits. First, a ceiling: every machine has a largest size, and once you reach it there is nowhere left to go. Second, cost: the biggest machines are disproportionately expensive, so each extra bit of capacity costs more than the last. Third, a single point of failure: if that one big machine fails, everything stops. Upgrades also usually mean a restart, so scaling up often means downtime.

Horizontal scaling means running many ordinary servers side by side behind a load balancer. There is no practical ceiling (large companies run thousands of servers for a single service), cost grows in step with capacity, and you can add or remove machines as traffic rises and falls, for example scaling down at 3am when nobody is awake.

It also improves availability in a way vertical scaling cannot. With ten identical servers sharing the traffic, one failure leaves 90% of your capacity, so most users never notice. With one big server, one failure leaves 0%. Teams often provision one spare on top of what peak needs (called N+1) so that losing any single machine has no visible effect.

The price is complexity. You now need a load balancer, a way to deploy to many machines, and logs and metrics gathered in one place, since you cannot log in to every server. And, most importantly, your servers must be stateless.

• -Logic (web and application) tier: scale out. It is the easy case, because these servers can be made stateless.
• -Database: scale up first. Splitting data across machines is difficult and hard to undo (a later chapter covers it), so use a bigger machine and read replicas for as long as that is enough.
• -In an emergency, scale up: it is the fastest relief because nothing needs redesigning. Then use the breathing space to plan the move to scaling out.
• -Plan for the ceiling: if your growth will outgrow the largest machine within a year or two, start the move to scaling out now, not on the day you hit it.

Neither is always right. A sound answer names the bottleneck first, then picks the cheapest move that relieves it.

The Wiki card below goes further into scaling up, including what the very largest machines cost:

And this one covers scaling out, including auto-scaling and the operational costs: