Dmitry Eroshenko

"That only takes 2ms"

In an interview I mentioned that I keep a big config object in process memory instead of reading it from Redis on every request. The interviewer countered: "a Redis request only takes a couple of milliseconds." He thought it was about latency. It never was. This is the explanation I didn't give him then, with real numbers and a benchmark you can run yourself.

Contents

  1. What the 2ms actually hides
  2. The setup
  3. The measurements
  4. When the config changes
  5. Run it yourself
  6. Key takeaways by the numbers

He was right about the latency: Redis on the same machine answers in a fraction of a millisecond, and I never argued with that. I didn't argue about the rest either: without numbers it would have been my word against his shrug, and he didn't seem interested in follow-up questions. So I saved the explanation for later. This is the later.

What the 2ms actually hides

One of the systems I wrote handles 2,730 requests per second on average. So that's the rate I benchmarked: the same config read, done two ways, at that system's everyday traffic — and nothing else. No business logic on top, even though in a real service it's always there, competing for the same cores.

Reading a config from Redis on every request costs almost no latency. What it costs, on every request, is everything around the latency:

  1. Network transfer. The config gets copied out of Redis into the Node.js process again and again — 6,417.65 MiB every minute (~6.3 GiB) at 2,730 requests a second, for data that hasn't changed since the last request.
  2. CPU for parsing. JSON.parse rebuilds the same object every time. At my system's rate that's 27.5% of the run's wall-clock time; at full load, 78.2%.
  3. RAM and garbage collection. Every request leaves behind a dead string and a dead object tree. At 2,730 req/s the GC pauses add up to 130.3 ms per minute, and the process holds 267 MiB versus 80.9 MiB.

This is not a latency optimization. What the in-process cache saves is RAM, garbage-collector work, and transfer bandwidth — and at this rate it's already visible on a dashboard.

The setup

  • Config: one JSON object, 41,083 bytes (~40 KiB).
  • Two ways to read it: from Redis (GET + JSON.parse on every request, over a connection pool) vs from memory (parse once, reuse the object).
  • Two traffic levels: 2,730 requests/sec — the average rate of the system this story comes from — and full load (50 concurrent workers, no rate limit, 60 s).
  • Node.js and Redis run on the same machine, over loopback — no network to blame.

The measurements

Every number below is copied verbatim from results/results.json, which the benchmark writes; the tables are printed from that file by node tables.js — no manual math in between.

One call, the Redis path split into its two steps:

Path mean p50 p95 p99 max
GET only 0.099 ms 0.096 ms 0.117 ms 0.135 ms 0.946 ms
GET + JSON.parse 0.172 ms 0.165 ms 0.195 ms 0.237 ms 1.395 ms

JSON.parse alone is 0.0662 ms per call. So per call the interviewer is right: caching wins you a fraction of a millisecond.

Now the same read at both traffic levels. parse is the share of the run's wall-clock time spent inside JSON.parse, GC pauses is the total garbage-collector pause time over the one-minute run, memory is the process RSS; CPU can exceed 100% of one core because the main thread and the GC/libuv threads add up:

throughput p99 latency transfer parse CPU GC pauses memory
from Redis — 2,730 rps 2,729.9 req/s 1.402 ms 6,417.65 MiB/min 27.5% of wall 42.3% of a core 130.3 ms/min 267 MiB
from memory — 2,730 rps 2,729.9 req/s 0.027 ms 0 MiB/min 0% of wall 2.5% of a core 10.3 ms/min 80.9 MiB
from Redis — full load 12,005.9 req/s 11.969 ms 28,223.24 MiB/min 78.2% of wall 104.1% of a core 569.4 ms/min 616 MiB
from memory — full load 12,800,546.4 req/s 0.005 ms 0 MiB/min 0% of wall 100.2% of a core 0 ms/min 198 MiB

At 2,730 requests a second the from Redis row moves ~6.3 GiB a minute through loopback sockets to deliver identical bytes, spends 27.5% of its time inside JSON.parse, and Redis itself adds another 19.05% of a core serving the same value over and over. At full load the process burns a whole core, 78.2% of it re-parsing JSON, and Redis's share grows to 55.72%. The from memory rows: zero transfer, zero parsing, almost no GC. (The huge req/s there isn't a real target — returning an object from memory is nearly free, so the row only shows the ceiling.)

When the config changes

The fair objection to an in-process cache is staleness: if every process keeps its own copy, what happens when the config changes? And it does change — someone edits it in the admin panel, which in practice happens once in a few weeks. In the system this story comes from, nothing dramatic follows — it's event-driven. Saving in the admin panel publishes a notification, every Node.js process receives it, re-reads the config once and swaps the object in memory. Redis stays the single source of truth; process memory just holds the hot copy. So a change costs one GET + JSON.parse per process once in a few weeks, while the reads happen thousands of times a second. There is no trade-off here.

Run it yourself

You need Node.js 18+ and Redis on 127.0.0.1:6379 (docker run --rm -p 6379:6379 redis). Download just-2ms-bench.zip, unzip it, then:

cd bench
npm install
node seed.js      # store the config in Redis
node run-all.js   # run everything, writes results/results.json
node tables.js    # prints the tables in this article from results.json

The run above: Node.js v25.2.1 + Redis 8.8.0 on Apple M5 Max, Darwin 25.5.0 (arm64), 64 GB RAM, 127.0.0.1 loopback.

Key takeaways by the numbers

  • At the system's average 2,730 requests a second, the Redis path moves ~6.3 GiB of traffic per minute for data that never changes.
  • 27.5% of the process's time goes into JSON.parse at that rate; at full load it grows to 78.2%.
  • GC pauses: 130 ms per minute versus 10 ms; at full load, 569 ms versus zero.
  • Memory footprint: 267 MiB versus 81 MiB.
  • And the Redis server itself spends ~19% of a core on top of that — ~56% at full load.

Latency numbers lie when you look at them one request at a time. Systems don't fail because of one slow call, they fail because of a thousand small ones stacked together.

So if you ever think a Redis read is fast, count everything it costs — not just the network latency.

← Back to blog