Sizing a Connection Pool With Math, Not Guesses

TL;DR

• Pool size isn't a guess: it comes from $L = \lambda W$ (Little's Law), your actual request rate and connection hold time
• PostgreSQL's real concurrency ceiling is capped by CPU cores, not by how big you set the pool
• As utilization $\rho$ approaches 1, queueing delay grows sharply, not linearly, small traffic spikes can cause outsized slowdowns
• Serverless and horizontally-scaled apps multiply this problem: a pooling proxy like PgBouncer belongs between your app and Postgres
• The formulas below take about ten minutes to run against your own metrics, and usually produce a smaller number than whatever's currently configured

The "just set it to 100" problem

Ask why a connection pool is sized the way it is, and the honest answer is usually "someone copied a number from a blog post." That number is often wrong in both directions. Too small, and requests queue behind a scarce resource. Too large, and the database spends more time context-switching between connections than doing work.

There's a small amount of math that replaces the guess. None of it is exotic.

Why connections are expensive to get wrong

PostgreSQL uses a process-per-connection model: every connection is a full operating system process, not a lightweight thread inside a shared pool. That buys strong isolation, one connection can't crash another, but it costs memory and setup time per connection. A baseline connection typically holds several megabytes of memory before it's run a single query, and forking a new process plus authenticating it adds latency a thread-based system wouldn't pay.

That's the specific reason PostgreSQL deployments are more sensitive to pool sizing than a generic "database pooling" guide would suggest.

Little's Law

Little's Law is a general queueing theory result: the average number of items in a system equals the arrival rate multiplied by the average time each item spends in the system.

$L = \lambda W$

Where $\lambda$ is the request arrival rate (requests per second) and $W$ is the average time each request holds a connection (seconds). $L$ is the average number of connections in use at any moment, the number you're actually trying to size for.

The same relationship explains why a coffee shop with an average of three people in line, one arriving every twenty seconds, needs to know how long each order takes, not just how many people showed up.

Example. An app receives $\lambda = 200$ requests per second. Each request holds a connection for an average of $W = 15\text{ms}$ (0.015s).

$L = 200 \times 0.015 = 3$

Three connections in use, on average. Not a hundred. If your pool is set to 100 for that workload, ninety-seven connections are sitting idle, each still costing the database memory and bookkeeping.

Add headroom for bursts, not for comfort

Average load isn't peak load. The right adjustment is a safety factor grounded in your actual traffic shape, not a round number that feels safe.

• Use $p95$ or $p99$ query time for $W$ instead of the average, if your traffic is bursty
• Multiply $L$ by a headroom factor, typically 1.5 to 3x, based on how spiky your traffic actually is
• Measure, then adjust. A pool size is a hypothesis until you've watched it under real load

How to actually measure λ and W

Little's Law is only as good as the numbers you feed it. Both are measurable from things you're probably already logging.

• $\lambda$: your application's request rate, pulled from access logs, a load balancer's metrics, or an APM tool. Use requests to endpoints that actually touch the database, not total traffic.
• $W$: query duration, from pg_stat_statements, a Prometheus histogram around your query calls, or PgBouncer's own SHOW STATS output if you're already running it. Use $p95$, not the mean, if your query times vary a lot.

Run the numbers once a quarter, or after a meaningful traffic change. A pool size set correctly at launch can be wrong six months later without anyone touching the config.

The classic PostgreSQL formula

Independent of your request rate, PostgreSQL itself has a practical ceiling on how many connections can do useful work at once, driven by CPU core count:

$\text{connections} = (\text{core\_count} \times 2) + \text{effective\_spindle\_count}$

For a typical small production box, 4 cores, SSD storage (effective spindle count of roughly 1):

$\text{connections} = (4 \times 2) + 1 = 9$

Nine. That's the number of connections that can be doing real work simultaneously before the database itself becomes the bottleneck, not your application layer.

This formula predates widespread NVMe storage, from when "effective spindle count" meant something closer to literal spinning disks. On modern SSD or NVMe-backed instances, that term is usually just set to 1. The core-count term is the one that still matters.

Why bigger pools make things slower, not faster

Utilization ties the two numbers together:

$\rho = \frac{\lambda W}{c}$

Where $c$ is your pool size. As $\rho$ approaches 1, queueing delay doesn't grow linearly, it grows sharply. A simplified single-connection approximation, where $\mu = 1/W$ is the service rate of one connection, makes the shape clear:

$W_q \approx \frac{\rho}{\mu(1-\rho)}$

Plotted out, the shape is unmistakable:

At $\rho = 0.5$, added wait time is small. At $\rho = 0.9$, it's roughly nine times larger. At $\rho = 0.99$, it's off the chart. This is why a pool that looks "almost big enough" during a load test can fall over in production the moment traffic ticks up 10 percent: you're not on the flat part of the curve anymore, you're on the part that goes vertical.

When pools multiply: the serverless problem

A single long-running server with one pool is the easy case. It gets harder the moment your app scales horizontally: more app instances, more serverless function invocations, each with its own pool, all pointed at the same PostgreSQL instance.

Ten app instances each configured for a pool of 20 isn't a pool of 20. It's up to 200 real connections competing for a database that might only handle 9 to 25 of them efficiently, per the core-count formula above. Serverless makes this worse: a burst of concurrent function invocations can each try to open their own connection at once, spiking well past whatever limit you thought you'd set.

The fix isn't a bigger max_connections value. It's a pooling proxy, PgBouncer or a managed equivalent, sitting between your application and Postgres. The proxy holds a small number of real database connections and multiplexes many more application-side connections onto them in transaction mode. Your app behaves as if it has a generous pool; Postgres only ever sees the number it can actually handle.

Where the queue actually forms

The queue you should be managing is in your application's pool, or in a proxy like PgBouncer, somewhere controlled and observable. The queue you don't want is an unbounded pile of connections opened directly against Postgres, each one consuming memory whether it's doing work or not.

What we actually set

The short version

A connection pool size is not a personality trait of your framework's default config. It's $L = \lambda W$, adjusted for burst headroom, capped by what your database's core count can actually execute concurrently. Measure $\lambda$ and $W$ from real traffic, run the formula, and you'll usually find the right number is smaller than whatever's currently set.