Caching in System Design

A cache is a faster, smaller copy. Disk is about a millisecond. RAM is about a hundred nanoseconds. Caching trades storage and invalidation for latency and load. The database remains the source of truth. The cache is a hint that can be wrong.

This note follows Evan's walkthrough. The statement model is Core SQL Concepts. HTTP and CDN reuse in this stack is Understanding Next.js in Depth. This note is the application cache.

Pattern Map

1. Why a cache exists

Name the bottleneck first. A cache without one is complexity with a TTL.

• Load. A read-heavy path that is taking Postgres down. Profile fetches, a homepage feed, a join that rebuilds on every request.
• Latency. A non-functional that the disk path cannot meet. Memory sits closer to the CPU; the query does not.
• Cost of compute. A personalized feed that joins posts, follows, likes. Cache the result for sixty seconds. Do not recompute it for every scroll. The denormalized feed is the cache; the tables stay normalized. Data Modeling in System Design.

Failure: dropping Redis in front of every table because "we always cache." The write path gets a dual-write. The read path gets stale keys. Postgres was never the problem.

2. Where it lives

Four places. The production default for application data is external.

Client → CDN → Hono → Redis → Postgres

• All Hono tasks share one Redis. Once one replica fills a key, the others reuse it. That is why external is the default.
• In-process is the right extra layer for something every request needs and almost never changes — feature flags, a country table — or as a shield in front of a hot Redis key. It is not a substitute for Redis in a fleet.
• A CDN is read-through at the edge. Origin is S3 or the API. The common interview and production use is images, video segments, static files. Understanding Next.js in Depth is the HTTP/CDN story in this repo. Browser storage is Local Storage, Session Storage, and Cookies.

Failure: an in-process map as the only cache, then scaling to two ECS tasks. Replica A has the new profile. Replica B still has the old one. Local cache is a performance hint, not a source of truth.

3. Cache-aside

The app owns the cache. Check Redis first. Hit: return. Miss: load Postgres, fill Redis, return. Only keys that were actually requested occupy memory.

async function getProfile(orgId: string, userId: string) {
  const key = `profile:${orgId}:${userId}`
  const hit = await redis.get(key)
  if (hit) return JSON.parse(hit)

  const row = await db.query.profiles.findFirst({
    where: and(eq(profiles.organizationId, orgId), eq(profiles.id, userId)),
  })
  if (row) await redis.set(key, JSON.stringify(row), "EX", 60)
  return row
}

• This is the production default. Redis does not need a write-through adapter. Hono talks to both. If Redis is down, reads fall through to Postgres — slower, still correct.
• The miss is the expensive path: DB + fill + return. That is the point of keeping the cache warm, not a reason to pick another architecture.
• The app owns TTL and invalidation. That control is why cache-aside wins over a library that hides the miss.

Failure: filling on miss without a TTL, then never deleting the key. The cache becomes a second, unbounded, stale database.

4. Write-through, write-behind, read-through

Cache-aside is how this stack reads. The other three change who writes, and when the database hears about it.

• Redis and Memcached do not do write-through natively. A library (or your own two writes) has to. Two writes is a dual-write: cache succeeds, DB fails, or the reverse. Perfect consistency across both is the same problem the outbox exists to avoid on the queue path.
• Write-behind belongs on analytics and counters where a lost batch is acceptable. View counts buffered in Redis and flushed to Postgres are this pattern. Invoices are not.
• Read-through is the CDN miss: edge fetches origin, stores, returns. For Hono + Redis, cache-aside is the same idea without the adapter.

Failure: write-behind on money. Treating Redis as a write-through engine it is not, then wondering why a crash lost the only copy of the write.

5. Keys, TTL, eviction

Memory is smaller than the dataset. Something has to leave. Name the key before naming the policy.

• The key is the identity of the cached value: profile:${orgId}:${userId}, not user:${userId}. A permission decision is never cached under a key that omits user or tenant. Isolation that lives only in Hono is one forgotten filter away from a leak — Building a Multi-Tenant Backend with Hono, Better Auth, Drizzle, and Postgres RLS.
• LRU evicts what has not been touched. It is the usual default. LFU evicts what is rarely touched, even if it was touched a second ago — right when a few keys dominate. FIFO is simple and rarely right. The LRU implementation is Common Algorithms.
• TTL is a freshness bound, not an eviction policy that replaces LRU. Sessions, feeds, API responses get a clock. LRU still decides what leaves when the set is full before the clock fires.
• Version the key when the payload shape changes (profile:v2:...). Invalidating "all profiles" by scanning Redis is a confession the key was too wide.

Failure: caching canEdit:${docId} without userId or organizationId. Every principal shares one decision. The cache did not leak. The key did.

6. Invalidation

Most systems read the cache and write the database. That window is stale data. There is no perfect fix. Freshness is a product choice.

Write Postgres → DEL key

Read → Redis
  hit  → Maybe stale
  miss → Postgres → SET key

• Invalidate on write. UPDATE the row, DEL the key. The next read misses and fills. Prefer delete over update-in-place: a concurrent miss can reload the old row and SET it back after your write.
• Short TTL when some staleness is fine. A newsfeed at 60 seconds. A profile picture at five minutes. Say the bound out loud.
• Accept eventual for feeds, counts, search. The person who just saved still needs read-your-writes — return the written entity in the POST, or read the primary for a short window. The cache is for everyone else.

Failure: SET the new value in Redis from the writer while a miss that started before the COMMIT writes the old row back. Delete the key. Let cache-aside refill.

7. Stampede

A popular key expires. For one second every request misses. One query becomes a hundred thousand. The database is the herd.

• Singleflight / request coalescing. The first miss loads Postgres. The rest wait and read the fill. Across replicas, a short Redis lock (SET key:lock NX EX 5) so only one process rebuilds.
• Cache warming. Refresh the homepage at 55s so the 60s TTL never fires. Warming helps TTL expiry. It does not help invalidate-on-write — that miss is the point.
• Stale-while-revalidate. Serve the old value past a soft TTL while one request refreshes. Block only past a hard TTL.
• TTL jitter. Do not expire every feed key on the same second. A random spread so related keys do not miss together.

Failure: stampede plus caching a transient failure for a long TTL — a dependency 500 stored as "not found". The next minute every client is sure the profile does not exist. Stampede is "every client was synchronized on expiry."

8. Hot keys

One key takes almost all the traffic. The cluster hit rate looks fine. One shard is on fire. Caching scales reads. It does not make Taylor Swift infinite.

• Replicate the hot key across shards so Hono can pick any replica. The rest of the keyspace stays partitioned.
• In-process in front of Redis for the handful of keys that would otherwise saturate a node. The app memory absorbs repeats; Redis sees a miss storm only on cold start or eviction.
• A hot key is the same shape as a hot row. Redis did not create it. It concentrated it.

Failure: adding Redis and calling the read path solved because p99 dropped — except for one profile: key that now melts a single node at peak. The bottleneck moved. It did not vanish.

9. What not to cache

Not everything that is slow should be remembered.

• Secrets and session tokens in a shared cache without the same controls as the source. Crash dumps, KEYS *, a replica that should not see them.
• Authorization under a key that omits the principal. Cache a rendered public document. Do not cache "may this user edit it."
• Private HTML and JSON on a shared CDN. Vary is required and often still wrong. Personalized or tenant-scoped payloads stay off the edge. The Next.js note is the Vary and Cache Components path.
• Negative transients. A 404 that is a 404 can be cached briefly. A 503 cannot.

Failure: a CDN that caches a JSON response meant to be private, or an in-process map that grows with every organizationId until the task OOMs. Memory does not leak because Redis has LRU. It leaks because this process held a reference.

Recap Q&A