Capacity Planning

Capacity planning answers a question most teams can't actually answer: how much traffic can this system take before it degrades, and how far away are we from that point? Not "what's our CPU usage" — that's a symptom — but the load at which latency leaves acceptable bounds, and the margin between there and now.

The reason it's worth doing deliberately is that systems don't degrade gracefully. As utilization approaches capacity, queueing delay grows non-linearly, so a system running comfortably at 70% can fall over at 90% with no warning in between. Teams that discover this during a traffic spike learn that the metric they were watching (CPU) was not the metric that mattered (queue depth, connection pool saturation, or a database at its IOPS limit).

The other reason is cost. Capacity planning is the discipline that distinguishes "we're paying for 3× headroom because we measured that we need it" from "we're paying for 3× headroom because nobody knows what we need."

TL;DR

Core Concepts

Find the knee, not the breaking point

The knee is where the queueing term in ρ/(1−ρ) starts to dominate. Below it, added load costs you a little latency; above it, added load costs you a lot. Load testing to the point of failure tells you where the cliff is; load testing to find where latency departs from flat tells you where to plan.

The USE method

Brendan Gregg's checklist, applied per resource. It's the fastest way to find a bottleneck because it's systematic rather than intuitive.

Saturation is the metric that warns you. Utilization at 100% might be fine — a batch job should saturate CPU. Saturation means work is waiting, which is queueing, which is where your latency went. A connection pool at 100% utilization with zero waiters is efficient; the same pool with 40 threads waiting is your outage.

The binding constraint

Adding CPU to this system does nothing. Common binding constraints, roughly in order of how often they're the real answer and how often they're overlooked:

The stateless-app-plus-shared-database shape deserves emphasis: scaling the app tier is easy and moves the constraint to the database, which is hard. Capacity planning that stops at the app tier plans for the wrong thing.

Headroom

The N+1 arithmetic is worth doing explicitly, because it's frequently missed:

Forecasting

The value of modelling this way is that it survives a product change. When someone proposes a feature adding three queries per page view, you can compute the capacity impact instead of guessing.

Autoscaling is not a capacity plan

Two things to know for any autoscaled service: what the reaction time actually is (measure it — from traffic arriving to new capacity serving), and what happens at maxReplicas. The second is a capacity decision disguised as a configuration value, and the answer should be load shedding rather than collapse.

Load Testing for a Real Number

Details that determine whether the result is usable:

The most common way a load test produces a useless number: hitting one cheap cached endpoint with a fixed VU count against a small dataset. It will report a throughput figure many times higher than reality.

Best Practices

Measure the knee, don't estimate it

Everything else in capacity planning depends on this one number, and it cannot be derived from a spec sheet. Load test to find where latency departs from flat, record it, and treat it as a property of the system that changes when the system changes.

Track saturation, not just utilization

Queue depth, waiting threads, connection pool waiters, run queue length. Saturation rises before latency does, which makes it the metric worth alerting on. Utilization tells you how busy something is; saturation tells you whether that's a problem.

Find the binding constraint before adding capacity

Scaling the wrong tier costs money and changes nothing. Run through the USE checklist per resource per tier, and expect the answer to be the database, a connection pool, or a downstream rate limit rather than CPU.

Do the N+1 arithmetic

If losing one instance pushes the rest past the knee, you don't have redundancy — you have a correlated failure waiting for a bad deploy. Compute the post-failure utilization explicitly and size for it.

Forecast from business drivers

Users, orders, tenants, events — the things product and sales can actually forecast. Then measure requests per driver unit and resources per request. Extrapolating a CPU graph gives you a number nobody can reason about and that breaks the moment the product changes.

Know your autoscaler's reaction time and its ceiling

Measure the delay from traffic arriving to new capacity serving requests. Then decide what should happen at maxReplicas — load shedding with a clear error is a better outcome than every request timing out.

Re-measure after significant changes

A new feature, a framework upgrade, a schema change, or a dependency addition can move the knee substantially. A capacity number from six months and forty deploys ago is a guess.

Plan for the peak, and know its shape

Daily peak, weekly peak, and the annual event (a sale, a launch, a season). Some workloads have a peak-to-trough ratio of 10×, which changes both the capacity target and whether autoscaling is worth the complexity.

Common Mistakes

Planning against CPU utilization

Load testing with a fixed VU count

Testing against a small dataset

Ignoring the downstream

Treating autoscaling as the plan

Extrapolating resource graphs

FAQ

What utilization should I target?

50–70% of the measured knee for a latency-sensitive service, and higher for batch or asynchronous work where queueing delay doesn't reach a user. The specific number should come from your own arithmetic: what does losing one instance do, how fast can you scale, how spiky is your traffic, and how much latency degradation is acceptable during a failure. Anyone quoting a universal target hasn't done that arithmetic.

How do I plan capacity for a database?

Differently and more carefully, because scaling it is slow and often requires downtime. The constraints are usually connections (use a pooler like PgBouncer), IOPS (check whether your volume is burstable and whether you're consuming credits), buffer cache hit ratio (a working set exceeding memory changes performance discontinuously), and lock contention. Plan further ahead than for stateless tiers, and know your read-replica and sharding path before you need it. See PostgreSQL Performance and Database Sharding.

Is capacity planning still necessary with cloud autoscaling?

Yes, and the questions shift rather than disappear. You still need to know the knee (to set scaling targets), the reaction time (to know what a spike costs), the ceiling (to know what happens at the limit), the account quotas (which are a real wall), and the downstream constraints (which don't autoscale). Autoscaling handles variation within a planned envelope; it doesn't tell you what the envelope is.

How far ahead should I forecast?

Match it to your lead time for adding capacity. Stateless services on a cloud provider: a quarter is usually plenty. Databases and stateful systems requiring migration or resharding: two to four quarters, because the work takes that long. Anything with hardware procurement or quota increases: longer still. Forecast to the horizon where you'd need to start acting today.

What's the difference between capacity planning and performance optimization?

Optimization reduces the resource cost per request, moving the knee to the right. Capacity planning determines how much capacity you need given the current cost per request. They interact: a 2× optimization halves the hardware needed for the same load, and is often cheaper than the hardware. Do the arithmetic both ways — sometimes the answer is genuinely "add servers," and sometimes one query fix is worth a year of growth.

How do I plan for a spike I can't predict?

Layer the defenses rather than trying to provision for it. Static headroom absorbs the small ones. Autoscaling absorbs the medium ones, within its reaction time. Load shedding with prioritization handles what's left — rejecting low-value traffic fast is far better than serving everything slowly, and it's the only thing that works when a spike outpaces provisioning. A queue in front of asynchronous work turns a spike into a backlog, which is a much better failure mode. See Rate Limiting.

Related Topics

References