From Modulo to Hash Ring: Scaling a Node.js Cache Fleet Without Miss Storms

A cache cluster that has run happily for months can take the database down within minutes of a routine change: adding one node. The culprit is usually a single line of client code that picks a server with hash(key) % N. This guide explains exactly why that line fails, compares the four serious alternatives, builds a production-quality consistent hash ring in Node.js, and lists the operational traps the algorithm itself will not protect you from.

The outage pattern

Picture a healthy fleet of four cache nodes with a 94% hit rate. Traffic is climbing ahead of a seasonal peak, so an engineer adds a fifth node. It is a two-line configuration change, rolled out carefully in the middle of a working day. About a minute and a half later the database is pinned at 100% CPU and the site is unavailable.

No one did anything careless. The client simply routed keys the way it always had:

const node = nodes[hash(key) % nodes.length];

That expression spreads keys very evenly, so it looks correct. The trouble is what it does the instant nodes.length changes, and that is where the rest of this guide begins. The discussion covers four separate problems, weighs the real alternatives, and then builds and measures a ring in Node.js.

Four problems hiding behind one idea

Consistent hashing is often introduced as a single trick. In practice it answers four distinct problems, and an implementation that handles only the first will still fail in production because of the other three.

Problem 1: changing N moves almost every key

With hash(key) % N, changing N does not relocate a few keys. It relocates nearly all of them.

It helps to work through the numbers. A key with hash 1,000,003 maps to node 3 with % 4, and happens to map to node 3 with % 5 as well. A key with hash 1,000,004 maps to node 0 under % 4 and node 4 under % 5. The two mappings have no relationship to each other, so a key stays where it was only by accident, with a probability of roughly 1 in N.

Measured across a million keys, the pattern is stark: going from 8 to 9 nodes moves 88.93% of keys, and on a hundred-node cluster, adding one node invalidates about 99% of the cache.

Look at which way that trend points. The larger your system grows, the more destructive each scaling step becomes. It is a failure that lies in wait until the business is succeeding.

Each relocated key is a miss, each miss is a database query, and they all arrive within seconds. A database provisioned for the 6% of reads that normally miss suddenly receives close to all of them.

Problem 2: the same reshuffle, unplanned

At least the first problem happens when you decide to scale. The second is the same event triggered by a failure, at the least convenient moment.

A node exhausts its memory, a host gets terminated, or a network partition hides one node from half the fleet. The node count falls from 8 to 7, and each client, acting on its own, remaps about 87% of its keys onto the remaining nodes.

The situation is now grim: 12.5% of cache capacity is gone, the database faces an 87% miss storm, and the seven survivors take on the lost node's traffic while being refilled with almost every key. A frequent result is that a second node collapses, forcing another full remap, which then topples a third.

Modulo routing turns one node failure into a correlated, cluster-wide failure that feeds on itself. That cascade, not a cold cache, is the real danger.

Problem 3: a naive ring is badly unbalanced

The obvious fix for problem 1 is to put nodes and keys into one numeric space and give each key to whichever node follows it going clockwise. That is the essence of consistent hashing, and it does eliminate the mass reshuffle.

Implemented naively, though, it balances load poorly. Nodes sit wherever their hashes land, so the gaps between them are random, and random gaps are rarely equal. With four nodes placed by one hash each, one measurement had a single node owning 45% of the keyspace and another only 13%, a 3.4x spread with no bug anywhere.

The imbalance is also persistent. It derives from the node names themselves, so the same node, say cache-04, stays hot until it is renamed, and anyone who investigates finds code that is working exactly as written.

At larger scale it gets worse. With one ring point per node across 8 nodes, the busiest node carried 434% of its fair share and the lightest carried 3.7%. That is effectively one overloaded server and seven idle ones.

Problem 4: every client must agree

The least visible problem is about authority. Something has to map a key to a node, and it must give the same answer on every machine that asks.

A coordinator service holding the authoritative shard map is one option. Then either every lookup costs a network round trip, or clients cache the map and you need a way to invalidate it. Should two clients hold different versions of the map, even for a moment, one may write user:42 to node A while another reads it from node B. Nothing is lost, which is arguably worse: there are now two plausible values.

What you want is a mapping that is a pure function of the key and the current membership list. No coordinator, no lookup service, no shared state: each client does the same arithmetic and reaches the same result. Consistent hashing provides exactly that, which is why it wins over a lookup table despite the table's greater flexibility.

A concrete scenario and the options

To make the comparison concrete, consider this system.

The system. An e-commerce API caches session and profile data in a pool of Redis nodes. Peak load is about 40,000 lookups per second over 25 million keys, with a 94% hit rate. The database only survives because it sees the 6% that miss. (For a refresher on the caching patterns themselves, see Redis caching fundamentals.)

The requirements:

1. Grow from 8 to 12 nodes before a sale without triggering a miss storm
2. Survive the loss of a node with a limited, tolerable blast radius
3. Keep load even, with no node above roughly 120% of its fair share
4. Keep any coordinator off the read path
5. Account for mixed hardware: some nodes have 64GB, others 16GB, and they should not carry the same load

Several algorithms can meet some or all of these. They differ in meaningful ways, and a poor choice is expensive.

Option A: modulo hashing

hash(key) % N gives perfect balance, costs one instruction and uses no memory.

It fails requirements 1 and 2 outright. It deserves a mention because it is what everyone writes first, and it behaves perfectly until the day it does not.

Choose it when N truly never changes, such as splitting a batch job across a fixed number of workers or sharding inside a single process.

Option B: a coordinator and a lookup table

Keep an explicit mapping from key ranges to nodes in a store such as etcd or ZooKeeper. Systems like Vitess and HBase work roughly this way.

The benefit is genuine: complete control. You can relocate a single hot shard, rebalance one range at a time while watching metrics, or pin a particular tenant to particular hardware. No hash-based approach offers any of this, and past a certain scale you will want it.

The price is equally genuine: a consensus system to run, the challenge of keeping every cached copy of the map current, and a mandatory dependency on the read path.

Choose it when you are moving durable data rather than disposable cache entries, and you need to control migration instead of letting everything shift in one go.

Option C: rendezvous hashing (HRW)

Highest random weight hashing scores each key against every node and picks the winner:

function rendezvous(key, nodes) {
  let best = null, bestScore = -1;
  for (const node of nodes) {
    const score = mix(hash(key), hash(node));
    if (score > bestScore) { bestScore = score; best = node; }
  }
  return best;
}

That is the entire algorithm. There is no ring, no virtual nodes, no sorted structure and nothing to rebuild when membership changes.

On the two metrics that matter most it also beats a ring. Going from 8 to 9 nodes moved 11.09% of keys, against a theoretical minimum of 11.11%, and balance was nearly perfect without any tuning.

The drawback is O(N) work per lookup, because every key is hashed against every node, and that cost climbs quickly as the fleet grows.

Choose it when you have fewer than about 30 nodes. Many teams run around eight cache nodes and would be best served by rendezvous hashing: it is simpler to write and reason about, and it balances better. The ring is the better-known answer, not automatically the better one.

Option D: jump consistent hash

This algorithm, published by Google in 2014, fits in about ten lines, needs no memory, balances almost perfectly and moves the minimum number of keys.

Its limit is structural. It maps a key to a bucket number in [0, N) and has no notion of node identity. Buckets can only be appended or dropped at the end of the range; there is no way to take node 3 out of the middle while keeping everything else stable.

Choose it when buckets are interchangeable and only their count changes, such as sharding a dataset across a resizable worker pool. It is a poor fit when specific named servers join and leave, which is precisely how cache nodes behave.

Option E: a hash ring with virtual nodes

This is the classic design. Nodes and keys share one circular address space, and a key belongs to the first node found clockwise from it.

Choose it when the fleet is large enough that the linear cost of rendezvous lookups becomes painful, and you also need weighted nodes and the ability to remove any node.

Picking one for the scenario

Measured on an 8 to 9 node change over one million keys, the alternatives other than modulo all move close to the theoretical minimum, and the ring's balance depends heavily on how many virtual nodes each server receives. For this e-commerce system, with mixed hardware, named nodes that can fail and a fleet expected to pass 30 nodes, the ring is the right choice. The rest of this guide builds it properly.

How the ring works

Set aside arrays and remainders. Imagine a circle numbered from 0 to 2³² − 1 that wraps around at the top.

Two rules make up the whole algorithm:

1. Hash every node name onto the circle, so cache-01 sits wherever its hash puts it.
2. Hash every key onto the same circle, then move clockwise. The first node you reach owns the key.

The key insight is that nodes and keys share one address space. Everything else follows from that, including why adding a node is inexpensive.

Why membership changes stay local

Place a new node on the circle and it falls between two existing ones. It takes over only the arc between itself and its counter-clockwise neighbor.

Keys outside that arc are unaffected and reach their previous owner exactly as before. The newcomer takes about 1/(N+1) of the circle on average, so that share of keys moves. Measured on an 8 to 9 change it was 11.06%, against a floor of 11.11%, compared with 88.93% for modulo on the identical change and key set.

Removal works in reverse: the departed node's arc passes to its clockwise successor. Shrinking from 8 to 7 nodes moved 12.60% of keys, close to the theoretical 12.50%. The impact is bounded and survivable, and the other six nodes are untouched.

Virtual nodes fix the imbalance

Back to problem 3: four nodes at four random positions produce very uneven arcs.

The remedy is surprisingly simple: do not place each node only once. Place it 160 times under 160 derived names like cache-01#0 and cache-01#1. Every derived name lands somewhere different, so every physical node owns 160 small scattered arcs instead of one big one, and the law of large numbers evens things out.

Measured over one million keys on 8 nodes, balance improves steadily as the replica count rises. 160 is the usual default because that is roughly where the improvement curve flattens, yet 500 is still noticeably better. A ring point needs roughly 12 bytes (a 4-byte position plus an 8-byte owner reference), so eight nodes at 500 replicas take under 50KB. If balance matters more to you than that memory, go higher; it is a calculation few teams bother to make.

Virtual nodes also make weighting essentially free. A node with twice the memory gets twice the points and so roughly twice the traffic. With weights of 4:4:1:1, the measured split was 40.6%, 41.1%, 9.4% and 9.0%, against an ideal of 40/40/10/10.

Implementing the ring in Node.js

The implementation comes down to four steps: hash the virtual node names, place them, sort them, and use binary search to find a key's owner. The class below keeps membership in a Map, rebuilds the sorted arrays when membership changes, and exposes get(key) for lookups.

export class ConsistentHashRing {
  #positions = new Uint32Array(0); // sorted ring positions
  #owners = []; // owners[i] owns #positions[i]
  #nodes = new Map(); // id -> { weight, points }

  constructor({ replicas = 160, hash = defaultHash } = {}) {
    if (replicas < 1) throw new RangeError("replicas must be >= 1");
    this.replicas = replicas;
    this.hash = hash;
  }

  addNode(id, weight = 1) {
    if (typeof id !== "string" || id.length === 0)
      throw new TypeError("node id must be a non-empty string");
    if (weight <= 0) throw new RangeError("weight must be > 0");
    if (this.#nodes.has(id)) return this;
    this.#nodes.set(id, {
      weight,
      points: Math.max(1, Math.round(this.replicas * weight)),
    });
    this.#rebuild();
    return this;
  }

  removeNode(id) {
    if (this.#nodes.delete(id)) this.#rebuild();
    return this;
  }

  #rebuild() {
    const pairs = [];
    for (const [id, { points }] of this.#nodes) {
      for (let i = 0; i < points; i++)
        pairs.push([this.hash(`${id}#${i}`), id]);
    }
    pairs.sort((a, b) => a[0] - b[0]);
    this.#positions = Uint32Array.from(pairs, (p) => p[0]);
    this.#owners = pairs.map((p) => p[1]);
  }

  /** Index of the first ring point >= h, wrapping to 0. */
  #successor(h) {
    const pos = this.#positions;
    let lo = 0,
      hi = pos.length;
    while (lo < hi) {
      const mid = (lo + hi) >>> 1;
      if (pos[mid] < h) lo = mid + 1;
      else hi = mid;
    }
    return lo === pos.length ? 0 : lo;
  }

  get(key) {
    if (this.#positions.length === 0) return null;
    return this.#owners[this.#successor(this.hash(key))];
  }
}

Three design choices deserve attention.

Parallel arrays instead of an array of objects. Storing positions in a Uint32Array keeps the binary search working over compact, contiguous memory that stays warm in the CPU cache. With 8 nodes and 160 replicas there are 1,280 points, about 5KB, and a lookup takes around 11 comparisons.

The wrap-around in lo === pos.length ? 0 : lo. A key that hashes beyond the last point belongs to the first node on the circle. Omitting that one condition is the most common bug in homemade rings: the result is correct for almost every key and inexplicably wrong for the few near the top of the range.

Rebuilding on membership change, not on lookup. Membership changes rarely while lookups happen tens of thousands of times a second, so sorting 1,280 entries now and then costs nothing meaningful.

Replication, meaning finding the next few owners for a key, is a clockwise walk that collects distinct physical nodes. The word "distinct" matters, because neighboring points on the ring often belong to the same server:

getReplicas(key, count = 1) {
  const n = this.#positions.length;
  if (n === 0) return [];
  const wanted = Math.min(count, this.#nodes.size);
  const out = [];
  const start = this.#successor(this.hash(key));
  for (let step = 0; step < n && out.length < wanted; step++) {
    const owner = this.#owners[(start + step) % n];
    if (!out.includes(owner)) out.push(owner);
  }
  return out;
}

Notice that wanted is capped at the number of physical nodes, so asking for more replicas than servers cannot loop forever, and the walk stops after one full lap in any case.

Choosing the hash function carefully

Many tutorials skip this part, and it holds the most instructive result of the whole exercise.

Most ring implementations default to MD5. It works but is slow: a ring using it reached only 423,000 lookups per second, and profiling showed that nearly all the time was spent inside MD5.

Replacing it with FNV-1a, a fast non-cryptographic hash, raised throughput about 15x. Balance, however, collapsed: the standard deviation of per-node load rose from 10.4% to 30.7%.

Dumping the computed positions for a few virtual nodes reveals the cause:

cache-01#0 → 4037809751
cache-01#1 → 4021032132
cache-01#2 → 4071364989
cache-01#3 → 4054587370
cache-01#4 → 3970699275

All of these land in a narrow band around 4.0 billion. FNV-1a has weak avalanche behavior, meaning that similar inputs produce similar outputs. Virtual node names differ by only a suffix, so rather than scattering 160 points around the circle, each node stacks them in one tight cluster. Optimizing for speed quietly brought problem 3 back.

The fix is to run FNV's output through a bit-mixing finalizer, the final step of MurmurHash3:

function fnv1a(str) {
  let h = 0x811c9dc5;
  for (let i = 0; i < str.length; i++) {
    h ^= str.charCodeAt(i);
    h = Math.imul(h, 0x01000193);
  }
  return h >>> 0;
}

// Scrambles the bits so near-identical inputs land far apart.
function fmix32(h) {
  h ^= h >>> 16;
  h = Math.imul(h, 0x85ebca6b);
  h ^= h >>> 13;
  h = Math.imul(h, 0xc2b2ae35);
  h ^= h >>> 16;
  return h >>> 0;
}

export const defaultHash = (str) => fmix32(fnv1a(str));

fmix32 alternates shifts, XORs and multiplications so that a change in any input bit spreads across all output bits. Math.imul performs true 32-bit integer multiplication, and >>> 0 converts the result back to an unsigned 32-bit number that fits the Uint32Array. For about ten extra operations, the combined hash balanced better than MD5 while running roughly 13x faster.

The broader lesson applies well beyond hashing: when you replace a component with a faster one, measure the property you were not optimizing. FNV-1a is a perfectly respectable hash; it is simply the wrong hash for this job, and nothing in its description warns you.

Turning the ring into a cache client

A ring alone is not a cache client. Production code must cope with failures, and the ring offers a tidy strategy: fall through to the next node clockwise.

The wrapper below takes a map of named clients, builds the ring from them, and on each get tries the first failoverDepth owners in order. A node that throws is marked down for downtimeMs and removed from the ring, then re-added once its penalty expires.

export class ShardedCache {
  #ring; #clients; #down = new Map();

  constructor(clients, { replicas = 160, failoverDepth = 2, downtimeMs = 10_000 } = {}) {
    this.#clients = new Map(Object.entries(clients));
    this.#ring = new ConsistentHashRing({ replicas });
    for (const id of this.#clients.keys()) this.#ring.addNode(id);
    this.failoverDepth = failoverDepth;
    this.downtimeMs = downtimeMs;
  }

  #markDown(id) {
    this.#down.set(id, Date.now() + this.downtimeMs);
    this.#ring.removeNode(id);
  }

  #reviveExpired() {
    const now = Date.now();
    for (const [id, until] of this.#down) {
      if (now >= until) { this.#down.delete(id); this.#ring.addNode(id); }
    }
  }

  async get(key) {
    this.#reviveExpired();
    for (const id of this.#ring.getReplicas(key, this.failoverDepth)) {
      try {
        return { value: await this.#clients.get(id).get(key), node: id };
      } catch {
        this.#markDown(id);
      }
    }
    return { value: null, node: null, allDown: true };
  }
}

Running this against four simulated nodes holding 10,000 keys and then killing one of them produced the following:

Keys per node:  cache-01 2213 | cache-02 2462 | cache-03 2923 | cache-04 2402

Killing cache-02...
  served from cache: 7538
  cache misses:      2462
  hard failures:     0
  healthy nodes:     cache-01, cache-03, cache-04

  24.6% of traffic became a miss.

That figure is your capacity plan. In a four-node fleet, one failure pushes an additional 24.6% of reads onto the database, while an eight-node fleet limits that to 12.5%. If the database cannot withstand 1/N of your read load arriving all at once, the real problem is database capacity, not caching, and consistent hashing has turned that problem from fatal into visible. Note also that there were zero hard failures: requests for the dead node's keys fell through to the next node and became ordinary misses.

Production pitfalls the algorithm does not cover

Several problems sit outside the algorithm and will hurt you regardless.

Ring version skew

The ring only helps if every client computes the same answer. Roll out a membership change gradually and, for a few minutes, half the fleet sees 8 nodes while the other half sees 9. The two halves disagree about roughly 11% of keys. For a cache that means a short dip in hit rate; for anything that accepts writes it means divergent data. Version the membership set, distribute it through a single channel, and report the version in your metrics so skew is something you observe rather than guess at.

Ejecting nodes too eagerly

Do not remove a node after one timeout. A flapping node that keeps leaving and rejoining causes a churn storm, because every transition moves 1/N of the keys. Require several consecutive failures within a time window before ejection, and re-admit cautiously. The example above uses a fixed ten-second penalty; production code should use exponential backoff and a health check before letting a node back in.

Hot keys are not balanced

Consistent hashing spreads keys evenly, but says nothing about requests. When a single product suddenly becomes wildly popular, its entry sits on one server, and that server overheats even though the ring is doing its job. Two remedies exist: a small in-process cache in front of the ring for the hottest keys, or the bounded loads variant of consistent hashing from Google research, which limits how much any node may take and sends the excess clockwise.

Remapping is not migration

Everything above assumes that losing a key only costs a cache miss. If the ring routes durable data, "11% of keys moved" means 11% of the data must be physically copied to its new node before it can be read there, typically with reads or writes going to both locations until the copy finishes. The ring identifies what has to move; it does nothing to move it. That is exactly why systems like Vitess rely on a coordinator: they need to control migration, not just calculate it.

The replica count is effectively permanent

Changing replicas from 160 to 500 shifts every ring position and reshuffles nearly all keys, which is as disruptive as a modulo change. Treat it as a one-time design decision made before you have live data, and if unsure, pick the higher value.

When a ring is the wrong tool

Part of engineering judgment is recognizing when the impressive solution is the wrong one.

• Fewer than about 30 nodes: use rendezvous hashing. It needs less code, balances better and has no replica count to tune. The ring's only advantage is O(log N) lookup, which is academic at that size.
• Interchangeable buckets where only the count changes: use jump hash, with ten lines of code and no memory.
• Durable data that needs controlled movement: use a coordinator. Only an explicit map lets you shift a single shard while monitoring its impact, something hashing cannot offer.
• N that truly never changes: % N is fine. Do not build machinery for a change that will not come.

A ring earns its place when nodes are named, heterogeneous, numerous and liable to fail. That describes a cache fleet almost perfectly, which is why so many distributed caches are built on one.

Key takeaways

• The core idea is small: put keys and servers in one address space, so a membership change disturbs only a neighborhood instead of everything.
• Modulo routing turns every scaling event and every node failure into a near-total cache flush, and the damage grows with cluster size.
• Rendezvous hashing is often the better choice for small fleets; reach for the ring when size, weighting and arbitrary removal all matter.
• Without virtual nodes, one server can carry several times its fair share. Pick the replica count up front, because changing it later reshuffles everything.
• A fast hash with weak avalanche can silently reintroduce imbalance; always measure distribution, not just speed.
• The loss of one node sends about 1/N of reads to the database. Size the database for that, version membership changes, eject nodes conservatively and handle hot keys separately.

The ring itself is only a few dozen lines of code. The engineering that keeps it reliable in production is everything around it.