Combine server count, CPU, memory, runtime model (thread-per-request vs async vs event loop), database topology, cache hit rate, and external API latency to simulate your system's max TPS and its first bottleneck.
Estimates are based on Little's Law (concurrency = TPS × latency). Thread-blocking runtimes hit their thread pool first; async runtimes hit CPU or sockets instead. Real systems vary — validate with a load test.
The backbone of this simulator is Little's Law: requests in flight = TPS × response time. Serve 1,000 requests per second at 100ms each and at any instant 100 requests are being processed.
Response time is modelled as three parts — pure CPU compute time + database query latency + external API latency. The longer that sum, the more threads, sockets and connections a given TPS keeps occupied.
Invert it and every resource yields a ceiling. With 200 threads and requests that hold a thread for 100ms, that pool tops out at 200 ÷ 0.1s = 2,000 TPS.
The calculation builds a list of per-resource TPS ceilings: CPU (cores ÷ compute time × instances), sockets (file-descriptor limit ÷ total response time × instances), the thread pool for thread-per-request runtimes, and the DB connection pool (whole load on a single DB, or split into writes and reads for master/slave).
The lowest ceiling caps the entire system and is reported as the bottleneck. Headroom elsewhere doesn't help — water never flows faster than its narrowest pipe.
Scaling decisions should follow the bottleneck. If the DB pool is the limit, adding app servers leaves max TPS unchanged and merely lengthens the queue in front of the database.
Thread-per-request (classic Spring MVC and friends) holds a thread even while waiting on external I/O. Waiting 50ms on the DB plus 300ms on an external API locks one thread for the full 350ms — so slow external calls drain the pool first.
Async runtimes (Virtual Threads, Go, Node.js) don't hold threads during I/O waits, so the same hardware reaches much higher TPS — and the limit shifts to CPU or sockets instead.
Node.js carries one extra clause: it has a single event loop, so CPU work beyond 20ms per request blocks the loop and every other request waits. The simulator raises a warning flag for that case.
Reads served from cache never reach the database. The model counts only read ratio × (1 − hit rate) as DB load — with 70% reads and an 80% hit rate, just 14% of all requests actually touch the DB.
So the more read-heavy the workload, the further cache pushes the connection-pool ceiling. A write-heavy service gains little from a better hit rate, because writes never pass through the cache.
With a master/slave topology, reads spread across replicas while writes all land on the master. A read bottleneck is solved by adding replicas; a write bottleneck yields to neither cache nor replicas.
It models handing slow external API calls to a queue and responding immediately: the external latency drops out of the request path, replaced by a 5ms enqueue cost.
With a 300ms external API the effect is dramatic on thread runtimes — each request holds its thread 300ms less, so the same pool sustains several times the TPS.
It also models overload absorption: traffic beyond capacity sits safely in the queue instead of piling up as waiting connections in server memory.
It isn't free. The response becomes "accepted" rather than a result, so it can't be used where callers need the answer immediately — that trade-off is an architecture decision, not a toggle.
Memory use is estimated as runtime baseline + in-flight connections × 1MB. Thread runtimes get a 512MB base plus 1MB per thread; async runtimes get 100MB per core.
In-flight connections are Little's Law again — TPS × response time. As responses slow down, the same traffic keeps more requests pinned in memory.
Running past capacity without an MQ stacks the excess as a backlog in memory. Once estimated use exceeds RAM, the reported bottleneck itself switches to OOM — the process dies before the throughput limit is ever reached.
This is a static, averages-based model. Traffic bursts, GC pauses, lock contention, connection-establishment cost and network bandwidth are not represented.
Real systems slow down non-linearly as they approach a bottleneck; here response time stays constant while resources last. The estimate is therefore close to an upper bound, and measured maxima usually come in lower.
Use it for direction: which resource saturates first, and whether cache, an MQ or replicas would actually help — before committing to a spec. Confirm final numbers against the real environment with tools like k6 or JMeter.
Each resource — CPU (cores ÷ compute time), sockets, thread pool, DB connection pool — has its own TPS ceiling; the lowest one caps the whole system and is reported as the bottleneck.
Thread-per-request (classic Spring MVC) holds a thread during external I/O waits, so the pool drains; Virtual Threads, Go, and Node.js don't hold threads during I/O, so the same hardware pushes much higher TPS.
Cache hits never reach the database, cutting connection-pool pressure. The more read-heavy your workload, the bigger the gain.
It models delegating slow external API calls to a queue, removing that latency from the request path — especially dramatic for thread-blocking runtimes.
What this tool bases its numbers on, and how far those numbers go.
Throughput is bounded by each layer in turn using Little's Law, concurrency = throughput × latency. A thread-per-request runtime is capped by its pool: max TPS = threads ÷ mean latency. Async runtimes are capped instead by CPU work per request or by socket and connection limits. Database capacity folds in the cache hit rate, and external API latency enters as added residence time. The reported bottleneck is whichever layer's ceiling is lowest.At 50 ms mean latency, serving 200 TPS needs 200 × 0.05 = 10 requests in flight. A 10-thread pool is therefore exactly saturated, and any latency increase turns into queueing rather than throughput.