Cache Stampede Causes & Mitigations: singleflight, Distributed Locks, and XFetch

Caching is one of the most effective ways to shield a relational database from repetitive read traffic. Yet in high-traffic systems, a paradoxical failure mode often strikes: the exact millisecond a popular cache key expires, hundreds or thousands of concurrent requests simultaneously miss the cache and slam the database with the exact same heavy query. This phenomenon is called a Cache Stampede (or dog-piling / thundering herd). This article walks through why cache stampedes happen and compares four proven mitigation strategies—in-process request coalescing (singleflight), distributed locking, probabilistic early expiration (XFetch), and TTL jitter.

1. An Everyday Analogy

Imagine a neighborhood bakery that keeps a display tray (the cache) stocked with fresh croissants every morning. As long as croissants are on the tray, customers grab one and leave in seconds. Now imagine the tray runs out during the morning rush, and all 100 customers standing in the store simultaneously shout into the kitchen (the database), "Bake me a new batch of croissants right now!" If the kitchen tries to start 100 separate ovens at once, the kitchen collapses—when in reality, baking one single batch to refill the display tray would have satisfied everyone waiting.

2. Concrete Failure Scenario in Numbers

Suppose computing the homepage "Popular Products" ranking requires a complex SQL aggregation that takes 500 ms, and the result is cached in Redis with a 5-minute TTL. Under a steady load of 1,000 requests per second, the moment that key expires at `05:00.000`, the very first request misses the cache and starts the 500 ms database query. During the 500 ms window before the first request finishes and repopulates Redis, 500 additional requests also see a cache miss and fire 500 identical expensive SQL queries at the database. Database CPU hits 100%, query latency stretches from 500 ms to 5 seconds, connection pools exhaust, and the entire service cascades into a timeout outage.

3. Four Mitigation Strategies Compared

Strategy	Core Mechanism	Coordination Scope
Request Coalescing (singleflight)	Only the first cache-miss caller queries the DB; concurrent callers in the same process await the same Promise/Future	Single process (per server instance)
Distributed Lock	Only the server instance that acquires a short Redis lock recomputes and refills the cache	Cluster-wide across all servers
Probabilistic Early Refresh (XFetch)	A single request probabilistically refreshes the key slightly before TTL expiration based on computation cost	Cluster-wide, lock-free
TTL Jitter	Adds a random time offset to each key's TTL so batch-warmed keys do not expire at the same second	Auxiliary defense for multi-key bursts

Notice that in-process coalescing (`singleflight`) operates per application server instance. If you run 10 Kubernetes pods, a cold expiry passes at most 10 queries to the database instead of 500—which is already a 98% reduction with zero network locking overhead. When even 10 queries are too expensive, pair `singleflight` with XFetch or a distributed lock.

4. Go Implementation: `golang.org/x/sync/singleflight`

In Go, the official extended package golang.org/x/sync/singleflight deduplicates concurrent function calls sharing the same string key so only one goroutine executes the closure while all other callers block and receive the shared return value:

type ProductService struct {
	group singleflight.Group
	cache Cache
	repo  Repository
}

func (s *ProductService) Popular(ctx context.Context) ([]Product, error) {
	if items, ok := s.cache.Get("popular"); ok {
		return items, nil
	}
	v, err, _ := s.group.Do("popular", func() (any, error) {
		items, err := s.repo.FindPopular(ctx)
		if err != nil {
			return nil, err
		}
		s.cache.Set("popular", items, 5*time.Minute)
		return items, nil
	})
	if err != nil {
		return nil, err
	}
	return v.([]Product), nil
}

Production gotcha with `ctx` cancellation: In the snippet above, if the very first caller cancels its HTTP request (`ctx` canceled), `s.repo.FindPopular(ctx)` aborts and returns `context.Canceled` to every waiting caller in the group! To prevent one impatient client from failing 99 other requests, pass a detached context with its own bounded timeout (e.g., `context.WithTimeout(context.WithoutCancel(ctx), 3*time.Second)`) inside the `singleflight.Do` closure.

5. NestJS / Node.js Implementation: Sharing In-Flight Promises

Because Node.js runs JavaScript on a single event loop per process, coalescing concurrent cache misses requires just a `Map>` that tracks in-flight promises and cleans up inside `finally()`:

@Injectable()
export class ProductService {
  private readonly inflight = new Map<string, Promise<Product[]>>();

  async popular(): Promise<Product[]> {
    const cached = await this.cache.get<Product[]>('popular');
    if (cached) return cached;

    const running = this.inflight.get('popular');
    if (running) return running;

    const task = this.loadAndCache().finally(() => {
      this.inflight.delete('popular');
    });
    this.inflight.set('popular', task);
    return task;
  }
}

6. XFetch: Optimal Probabilistic Early Expiration (VLDB 2015)

Proposed by Vattani, Chierichetti, and Lowenstein in "Optimal Probabilistic Cache Stampede Prevention" (VLDB 2015), the XFetch algorithm prevents the cache key from ever expiring during active traffic. Alongside the cached value, you store delta (how many milliseconds the last DB computation took) and the logical expiry timestamp. On every cache read, each server evaluates a single random inequality:

// beta defaults to 1.0; higher beta triggers earlier refresh
const shouldRecomputeEarly =
  !entry || now - entry.delta * beta * Math.log(Math.random()) >= entry.expiry;

Because `Math.log(Math.random())` is always negative, `- entry.delta * beta * Math.log(Math.random())` adds a positive lookahead jump scaled to how slow the query is (`delta`). When `expiry` is still minutes away, the probability of triggering is virtually zero; as `now` approaches `expiry` within a few multiples of `delta`, one lucky request triggers an early background refresh and extends `expiry`—while all other concurrent requests continue serving the still-valid cached value with zero lock contention!

Recomputing with a Distributed Lock + Stale-While-Revalidate

If you prefer explicit distributed locking when a cold cache miss occurs:

1. Read the cache; if fresh, return immediately.
2. On a miss, attempt a non-blocking Redis lock (`SET lock:popular NX PX 3000`).
3. The instance that wins the lock double-checks the cache, queries the DB, writes the new cache entry, and releases the lock.
4. Instances that fail to acquire the lock either serve a slightly stale backup value or wait briefly (`50–100 ms`) and re-read the cache (with a fallback circuit breaker so they don't hang forever).

7. Common Mistakes to Avoid

• Warming thousands of keys in a batch job with the exact same TTL: If a cron job populates 10,000 product keys with `EX 3600`, all 10,000 keys expire at the exact same second an hour later. Always add random jitter (e.g., `3600 + randomInt(0, 300)` seconds).
• Forgetting the Double-Checked Locking read after acquiring a lock: If 20 pods wait on a lock one after another without checking whether the previous lock holder already refilled the cache, they will still execute 20 sequential DB queries.
• Over-engineering distributed locks when accuracy isn't at stake: Cache regeneration is an efficiency optimization, not a financial ledger mutation. As explained in Redis Distributed Locks, if two servers occasionally compute a cache value twice during a network blip, nothing breaks.

8. Summary

References

• Vattani et al., Optimal Probabilistic Cache Stampede Prevention (VLDB 2015)
• Go Package Documentation - golang.org/x/sync/singleflight
• Wikipedia - Cache Stampede