"Make it faster" is not one problem - it's at least three, and they don't always move together. Latency is how long one request takes. Throughput is how many requests a system can handle per unit of time. And a bottleneck is whichever single stage in the pipeline is actually constraining one of those two, which is rarely the stage intuition points to first. Performance work that skips straight to "add more replicas" or "optimize this loop" without first pinning down which metric is actually broken, and where, tends to produce numbers that look different without actually being better.
This page is the mental model behind the rest of the section: what these terms precisely mean, how they interact under load, and the categories of bottleneck - event-loop, CPU, memory/GC, downstream dependency - that Performance Basics, Load Testing, and Memory & GC Tuning all assume you can already tell apart.
Latency (time per request) and throughput (requests per unit time) are distinct metrics, and a bottleneck is the single constrained stage limiting one of them - identifying which one is limited, and where, has to come before any fix.
Insight: Optimizing the wrong stage - or the wrong metric entirely - burns engineering time and can even make the metric you actually care about worse.
When to Use This Model: Investigating a slow endpoint, deciding what a load test should actually measure, reading a profiler's output with the right question in mind, and setting realistic SLOs before a launch.
Limitations/Trade-offs: Reducing latency and increasing throughput sometimes conflict directly - batching improves throughput by adding latency to individual requests - so "faster" needs a metric attached before it means anything.
Picture a highway. Latency is how long it takes one specific car to get from the on-ramp to the off-ramp - a single trip's duration. Throughput is how many cars pass a fixed point on that highway per hour - the road's total capacity, independent of any one driver's trip time. At low traffic, both numbers look great: cars move fast, and plenty of them get through. The interesting behavior shows up as traffic increases: throughput climbs steadily right up until the highway's narrowest point can't absorb any more cars per hour, and past that saturation point, every additional car doesn't raise throughput at all - it just sits in a growing queue, and latency for everyone already on the road climbs sharply.
That narrowest point is the bottleneck - the one stage of the whole system that's actually constraining capacity. Widening every other lane on the highway does nothing if the real bottleneck is a single-lane bridge three exits later; the fix has to target the actual constraint, not wherever intuition first looks. This is the central discipline performance work requires: measure first, identify which stage is actually saturated, and only then act - not the reverse.
The first trap in measuring latency is reaching for an average. An average of 1,000 requests where 950 take 20ms and 50 take 2 seconds still reports a "fast" number, because a handful of very slow outliers get diluted into a large pool of fast ones - but those 50 slow requests are exactly the experience some fraction of your users actually had. Percentiles fix this by asking a sharper question: p95 latency is the value below which 95% of requests fall, meaning the slowest 5% were worse than that number. Tracking p95 or p99 instead of the average is how you keep the tail - the requests users actually remember - visible instead of averaged away.
import { monitorEventLoopDelay } from "node:perf_hooks";const histogram = monitorEventLoopDelay({ resolution: 20 });histogram.enable();// Rising p99 loop delay with flat CPU usage means something is blocking// synchronously - not that the machine needs more compute.
In a Node service specifically, the bottleneck is rarely "the CPU is out of cycles" - it's more often one of a few distinct failure shapes that each need a different fix. Event-loop contention happens when synchronous work - a big JSON parse, a tight loop, a blocking crypto call - occupies the single JS thread long enough that every other pending request has to wait behind it, even ones with nothing to do with the slow one. CPU-bound work is the case where the loop itself is fine but a specific handler is doing genuinely heavy computation that no amount of concurrency helps, because it's bound by a single core's throughput. I/O wait looks like slowness but isn't compute-bound at all - the process is idle, waiting on a database or a downstream API, and the fix lives in that other system or in how many concurrent calls you allow, not in your code's efficiency. GC pauses show up as intermittent latency spikes uncorrelated with request complexity, caused by the garbage collector reclaiming memory rather than by the request itself doing more work.
Mistaking one of these for another wastes real effort: profiling CPU usage when the actual problem is a downstream API's p99 latency won't find anything, because the bottleneck was never in your process at all.
Load testing exists specifically to find the saturation point deliberately rather than discovering it in production. A test that ramps concurrency gradually - rather than firing a fixed number of requests all at once - reveals the shape of the highway: throughput rising linearly at low load, then flattening, then latency climbing sharply once a real constraint is hit. Load Testing covers building that ramp and reading the resulting curve; the crucial habit it enforces is testing at production-realistic concurrency, since a bottleneck that only appears above 200 concurrent connections is invisible to a single manual request no matter how carefully you time it.
Latency and throughput aren't always aligned goals, and conflating them leads to fixes that help one while quietly hurting the other. Batching several small operations into one larger one typically improves throughput - fewer round trips, better amortization of fixed costs - while making the individual request that got batched wait longer for its batch to fill, which worsens its latency. Neither choice is universally correct; it depends on whether your system is optimizing for a user staring at a spinner (latency-sensitive) or for total work completed per hour (throughput-sensitive), and that decision belongs to the product, not the profiler.
Garbage collection deserves particular attention in Node because its pauses are a source of tail latency that's easy to misattribute to "the code" when it's actually a memory-shape problem. A service allocating large short-lived objects on every request pressures the collector in a way that shows up as periodic latency spikes rather than a steady CPU cost - Memory & GC Tuning covers diagnosing that pattern specifically, and JSON Serialization Cost covers one of the most common sources of exactly this kind of allocation pressure in a typical JSON API.
Bottleneck category
Symptom
Where to look
Typical fix
Event-loop contention
Rising p99 loop delay, flat CPU, all endpoints slow together
monitorEventLoopDelay, flame graphs
Move blocking work off the main thread, chunk large synchronous operations
CPU-bound handler
High CPU on one route, unaffected by concurrency changes
Per-route CPU profiling
Algorithmic fix, offload to a worker thread
I/O wait / downstream dependency
Process idle, latency tracks a specific external call
Distributed tracing spans
Timeouts, caching, fixing the downstream system
Memory / GC pressure
Periodic latency spikes uncorrelated with request complexity
Heap snapshots, GC logs
Reduce allocation rate, tune heap size only after that
"A lower average response time means the service is faster for users." Averages hide the tail - a service can have a great average and a terrible p99, which is the number the unluckiest fraction of users actually experience.
"Adding more replicas always fixes latency." More replicas raise throughput capacity by handling more concurrent requests, but they do nothing for an individual request's latency if the bottleneck is a slow downstream call or GC pause each replica hits independently.
"Throughput and latency always trade off against each other." They're often independent - fixing an actual bottleneck (an unnecessary synchronous block, say) can improve both at once; they only trade off directly in specific cases like batching.
"A load test run on a laptop reflects production performance." Different hardware, network topology, and concurrent load patterns mean a local test mostly validates correctness, not the capacity or saturation point of the real deployment.
"If CPU usage looks fine, there's no bottleneck." Event-loop contention, I/O wait, and GC pauses can all cause serious latency problems while CPU utilization stays completely unremarkable.
What's the actual difference between latency and throughput?
Latency measures how long a single request takes; throughput measures how many requests the system completes per unit of time - a system can improve one without improving the other, so they need to be tracked and reasoned about separately.
Why do performance dashboards use p95/p99 instead of average latency?
Averages get diluted by a large number of fast requests, hiding the slower tail that a meaningful fraction of users actually experienced; percentiles report exactly how bad the slowest N% of requests were, which is closer to what "feels slow" means to a real user.
What does "saturation" mean in this context?
The point at which a system's throughput stops increasing no matter how much more load is applied, because some resource - CPU, a connection pool, a downstream dependency - is fully consumed; load beyond that point queues up and increases latency instead of raising throughput.
How do I tell whether my Node service's bottleneck is the event loop or something downstream?
Event-loop contention shows up as rising loop delay with flat CPU usage and every endpoint slowing together; a downstream bottleneck instead correlates specifically with calls to that one dependency and shows up clearly in a tracing span for that call, while the process itself stays idle.
Why does GC show up as a performance problem at all?
Garbage collection has to pause parts of the JS thread to reclaim memory, and a service with a high allocation rate - lots of short-lived objects per request - triggers that reclamation more often, producing latency spikes that don't correlate with any specific request's actual complexity.
Does more CPU always fix a performance problem?
No - it only helps if the bottleneck is genuinely CPU-bound computation; adding CPU does nothing for a service waiting on a slow database, blocked by a single synchronous call on the main thread, or paused by GC.
Can optimizing for throughput make latency worse?
Yes - batching is the clearest example: grouping several operations into one larger request improves overall throughput but makes the individual request that had to wait for its batch to fill slower, which is a deliberate and sometimes correct trade-off, not a bug.
Why does a load test need to ramp up concurrency instead of firing everything at once?
A gradual ramp reveals the actual shape of the system's capacity curve - where throughput plateaus and where latency starts climbing - while an instant spike mostly tests how the load generator and connection pool behave under a sudden burst, not the service's real saturation point.
Is a fast response in local development a reliable performance signal?
Not on its own - a single local request has no concurrent load competing for the event loop, no realistic network latency to downstream services, and no production-scale data volume, all of which are exactly what expose real bottlenecks.
What's the first thing to check when an endpoint is reported as "slow"?
Whether it's a latency problem (that endpoint specifically is slow) or a throughput problem (the whole system degrades under concurrent load) - the two point toward completely different bottleneck categories and different tools to investigate them.
Why is measuring before optimizing so heavily emphasized in Node performance work?
Because Node's bottleneck categories - event-loop contention, CPU-bound handlers, I/O wait, GC pressure - produce different symptoms and need different fixes; optimizing based on intuition instead of a profiler or trace risks fixing a stage that was never actually the constraint.