Caching

Why caching exists

Every time a client asks the Task API for its tasks, your code asks the database. The database is fast, but not free: each query crosses the network, uses a connection, and takes a few milliseconds. When the same list is asked for a hundred times a minute and changes twice a day, most of that work is wasted.

A cache is a small, fast store where you keep a copy of a result that was expensive to produce. The next time you need the same result, you read the copy instead of doing the work again.

To see the effect, here is a fake task database. It waits 20 milliseconds, like a real query, and counts how many queries it ran. It stands in for the repository you built in the database lesson, so every example on this page can run in your browser:

export interface Task {
  readonly id: number;
  readonly userId: string;
  readonly title: string;
  readonly done: boolean;
}

const wait = (ms: number) => new Promise<void>((resolve) => setTimeout(resolve, ms));

export class TaskDb {
  queries = 0;
  private readonly rows: Task[] = [
    { id: 1, userId: "ada", title: "Buy milk", done: false },
    { id: 2, userId: "ada", title: "Write report", done: true },
    { id: 3, userId: "linus", title: "Fix bug", done: false },
  ];

  async listTasks(userId: string): Promise<Task[]> {
    this.queries += 1;
    await wait(20);
    return this.rows.filter((task) => task.userId === userId);
  }

  async addTask(userId: string, title: string): Promise<Task> {
    this.queries += 1;
    const task: Task = { id: this.rows.length + 1, userId, title, done: false };
    this.rows.push(task);
    return task;
  }
}

import { TaskDb } from "./task-db.js";

const db = new TaskDb();

for (let request = 1; request <= 3; request++) {
  const tasks = await db.listTasks("ada");
  console.log(`request ${request}: ${tasks.length} tasks`);
}
console.log("queries:", db.queries);

request 1: 2 tasks
request 2: 2 tasks
request 3: 2 tasks
queries: 3

Three requests, three identical queries. With a cache, the first request runs the query and stores the answer. The next two read the copy. Two words you will see everywhere:

• A hit: the value was in the cache.
• A miss: it was not, so you have to do the slow work.

Install @zudojs/cache

In your task-api folder:

$ npm install @zudojs/cache

added 2 packages, and audited 74 packages in 2s
…

The … hides the same funding, audit and esbuild notes you saw when you created the project. The numbers depend on what you have installed so far. The package runs in the browser terminal too, so you can press Run in browser on every example here.

To run an example on your computer, save it in src/ next to task-db.ts and run it with tsx, for example npx tsx src/no-cache.ts. You should see the same output as the page.

Your first cache

A CacheService is the object you call. Behind it sits an adapter, the actual storage. The package ships an in-memory adapter, which keeps entries in a Map inside your process:

import { createCacheService, createMemoryCacheAdapter } from "@zudojs/cache";
import type { Task } from "./task-db.js";

const cache = createCacheService({
  adapter: createMemoryCacheAdapter({ maxEntries: 1000 }),
  config: { defaultTtl: 60_000 },
});

const tasks: Task[] = [{ id: 1, userId: "ada", title: "Buy milk", done: false }];
const saved = await cache.set("tasks.ada", tasks);
console.log(saved.success, saved.key);

const found = await cache.get<Task[]>("tasks.ada");
console.log(found.hit, found.value);

const missing = await cache.get<Task[]>("tasks.linus");
console.log(missing.hit, missing.value);

true zudojs:tasks.ada
true [ { id: 1, userId: 'ada', title: 'Buy milk', done: false } ]
false null

• maxEntries caps the cache at 1,000 entries. When it is full, the entry that was used least recently is dropped. A cache is allowed to forget.
• defaultTtl is how long an entry lives, in milliseconds. More on that below.
• get never returns the value on its own. It returns { hit, value }. Check hit first, because a cached value can itself be null or an empty list.
• The stored key is zudojs:tasks.ada. The cache added a prefix, zudojs, so its keys cannot clash with other data in a shared store such as Redis.

Keys and namespaces

A key is the name you store a value under. The cache builds the full key as prefix:namespace:key, with a colon between the parts. Because the colon has that job, it is not allowed inside your own key. Each part may only contain letters, digits, ., _ and -:

import { createKeyBuilder, isCacheError } from "@zudojs/cache";

const keys = createKeyBuilder({ prefix: "taskapi" });

console.log(keys.build("tasks.list"));
console.log(keys.build("tasks.list", { namespace: "ada" }));

try {
  keys.build("tasks:list");
} catch (error) {
  if (isCacheError(error)) {
    console.log(error.code, error.statusCode);
    console.log(error.message);
  }
}

taskapi:tasks.list
taskapi:ada:tasks.list
ERR_INVALID_INPUT 400
Invalid cache key part "tasks:list": parts must match /^[a-zA-Z0-9._\-]+$/ and must not contain the separator ":".

A bad key is a bug in your code, so it throws a CacheError with the code ERR_INVALID_INPUT and status 400. It is never treated as a miss.

Why so strict? If a key could contain a colon, a key such as "ada:tasks.list" would look exactly like the key tasks.list in the namespace ada. Someone who controls part of a key could then read or overwrite another user's entry.

A namespace groups keys. The same key in two namespaces is two different entries. In the Task API, every user sees only their own tasks, so the user id is a natural namespace:

import { createCacheService, createMemoryCacheAdapter } from "@zudojs/cache";

const cache = createCacheService({ adapter: createMemoryCacheAdapter() });

await cache.set("tasks.list", ["Buy milk", "Write report"], { namespace: "ada" });
await cache.set("tasks.list", ["Fix bug"], { namespace: "linus" });

console.log((await cache.get("tasks.list", { namespace: "ada" })).value);
console.log((await cache.get("tasks.list", { namespace: "linus" })).value);

console.log(await cache.clear({ namespace: "ada" }));
console.log(await cache.has("tasks.list", { namespace: "ada" }));
console.log(await cache.has("tasks.list", { namespace: "linus" }));

[ 'Buy milk', 'Write report' ]
[ 'Fix bug' ]
{ cleared: 1 }
false
true

A namespace is checked like a key part. An empty string or a wildcard such as "*" throws instead of quietly reaching every user's entries.

TTL: how long an entry lives

TTL means "time to live": the number of milliseconds an entry stays valid after you store it. When the time is up, the cache treats the entry as missing, and the next read goes to the database and gets fresh data. A TTL is your safety net against stale data: even if you forget to clear an entry, it cannot stay wrong for longer than its TTL.

import { createCacheService, createMemoryCacheAdapter, isCacheError } from "@zudojs/cache";

const wait = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));
const cache = createCacheService({ adapter: createMemoryCacheAdapter() });

await cache.set("tasks.ada", ["Buy milk"], { ttl: 100 });
await cache.set("settings", { theme: "dark" }, { ttl: null });

console.log("right away:", (await cache.get("tasks.ada")).hit);
await wait(150);
console.log("150 ms later:", (await cache.get("tasks.ada")).hit);
console.log("time left:", await cache.ttl("tasks.ada"), await cache.ttl("settings"));

await cache.set("tasks.linus", ["Fix bug"], { ttl: 10_000 });
const left = await cache.ttl("tasks.linus");
console.log("whole ms:", Number.isInteger(left), left !== undefined && left !== null && left <= 10_000);

try {
  await cache.set("tasks.ada", ["Buy milk"], { ttl: 0 });
} catch (error) {
  if (isCacheError(error)) console.log(error.code);
}

right away: true
150 ms later: false
time left: undefined null
whole ms: true true
CACHE_INVALID_TTL

The entry with a 100 ms TTL was gone 150 ms later. cache.ttl(key) returns the milliseconds left, undefined for a missing key, and null for an entry stored with ttl: null, which never expires. For a live entry you get whole milliseconds, a little under the TTL you set, because some time has passed since set.

The last line shows a common mistake: ttl: 0 does not mean "forever". It is rejected with CACHE_INVALID_TTL. Write ttl: null for an entry that never expires.

How long should a TTL be? It is a trade-off. Longer means more hits and less database work. Shorter means stale data disappears sooner. A task list that you also clear on every change (next sections) can live for minutes. Data you cannot clear reliably, such as an answer from another company's API, should live only as long as you can accept it being out of date. The default is 5 minutes, and the maximum is 24 hours.

getOrSet and the stampede

Almost all cache code does the same three steps: look in the cache; on a miss, do the slow work; store the result. getOrSet does all three:

import { createCacheService, createMemoryCacheAdapter } from "@zudojs/cache";
import { TaskDb } from "./task-db.js";

const db = new TaskDb();
const cache = createCacheService({ adapter: createMemoryCacheAdapter() });

for (let request = 1; request <= 3; request++) {
  const result = await cache.getOrSet("tasks.list", () => db.listTasks("ada"), {
    namespace: "ada",
    ttl: 60_000,
  });
  console.log(`request ${request}: ${result.value.length} tasks, cached: ${result.cached}`);
}
console.log("queries:", db.queries);

request 1: 2 tasks, cached: false
request 2: 2 tasks, cached: true
request 3: 2 tasks, cached: true
queries: 1

One query instead of three. result.cached tells you where the value came from.

Now imagine the entry has just expired and 50 requests arrive in the same millisecond. They all miss, so they all run the query at once. The database suddenly gets 50 identical queries: a cache stampede. On a busy site this can take the database down right when the cache was supposed to protect it.

getOrSet protects you. While one computation for a key is running, other callers for the same key wait for it instead of starting their own:

import { createCacheService, createMemoryCacheAdapter } from "@zudojs/cache";
import { TaskDb } from "./task-db.js";

const db = new TaskDb();
const cache = createCacheService({ adapter: createMemoryCacheAdapter() });

const load = () => cache.getOrSet("tasks.list", () => db.listTasks("ada"), { namespace: "ada" });

const results = await Promise.all([load(), load(), load(), load(), load()]);
console.log("answers:", results.map((r) => r.value.length));
console.log("queries:", db.queries);

answers: [ 2, 2, 2, 2, 2 ]
queries: 1

Promise.all, from the asynchronous JavaScript lesson, starts all five at the same moment. Five answers, one query.

Invalidation: when the data changes

Here is the bug every cache has at first. Ada adds a task, and her list does not show it:

import { createCacheService, createMemoryCacheAdapter } from "@zudojs/cache";
import { TaskDb } from "./task-db.js";

const db = new TaskDb();
const cache = createCacheService({ adapter: createMemoryCacheAdapter() });
const list = () => cache.getOrSet("tasks.list", () => db.listTasks("ada"), { namespace: "ada" });

console.log("before:", (await list()).value.length);
await db.addTask("ada", "Call the bank");
console.log("after: ", (await list()).value.length);

before: 2
after:  2

The database has 3 tasks for Ada, but the cache still holds the old list of 2, and it will keep serving it until the TTL runs out. Removing entries that are no longer correct is called invalidation. You have three tools:

• delete(key) removes one key you know.
• Tags: labels you attach when you store an entry. invalidateByTag(["tasks"]) removes every entry with that tag, without you listing the keys.
• Patterns: invalidateByPattern("tasks.*") removes every key that matches. * matches any characters, but never crosses the : between namespace and key.

Tags are the most useful, because one change often makes several entries stale. A new task changes the full list, the "open tasks" list and the count:

import { createCacheService, createMemoryCacheAdapter } from "@zudojs/cache";

const cache = createCacheService({ adapter: createMemoryCacheAdapter() });
const ada = { namespace: "ada", tags: ["tasks"] };

await cache.set("tasks.list", ["Buy milk", "Write report"], ada);
await cache.set("tasks.open", ["Buy milk"], ada);
await cache.set("tasks.count", 2, ada);
await cache.set("profile", { name: "Ada" }, { namespace: "ada", tags: ["profile"] });
await cache.set("tasks.list", ["Fix bug"], { namespace: "linus", tags: ["tasks"] });

console.log(await cache.invalidateByTag(["tasks"], { namespace: "ada" }));
console.log("ada list:", await cache.has("tasks.list", { namespace: "ada" }));
console.log("ada profile:", await cache.has("profile", { namespace: "ada" }));
console.log("linus list:", await cache.has("tasks.list", { namespace: "linus" }));

console.log(await cache.invalidateByPattern("tasks.*", { namespace: "linus" }));

{ cleared: 3 }
ada list: false
ada profile: true
linus list: true
{ cleared: 1 }

Three of Ada's entries were cleared. Her profile had a different tag and stayed. Linus's list had the same tag, but tags live inside a namespace, so his entry was not touched until the pattern call removed it.

Tags follow the same rules as key parts. An empty tag, tags: [""], throws ERR_INVALID_INPUT, the same code as a bad key.

The rule to follow: every code path that writes the data also invalidates its cache entries, right after the write succeeds. Create, update, delete, and any background job that changes tasks.

Cached objects are shared

The memory adapter stores the object you give it, not a copy. If any code changes that object later, every reader of the cache sees the change:

import { createCacheService, createMemoryCacheAdapter, JsonCacheSerializer } from "@zudojs/cache";

const shared = createCacheService({ adapter: createMemoryCacheAdapter() });
const task = { title: "Buy milk", due: new Date("2026-10-01T09:00:00Z") };
await shared.set("task.1", task);
task.title = "CHANGED BY ACCIDENT";
console.log((await shared.get<typeof task>("task.1")).value?.title);

const copying = createCacheService({
  adapter: createMemoryCacheAdapter(),
  config: { serializer: new JsonCacheSerializer() },
});
const task2 = { title: "Buy milk", due: new Date("2026-10-01T09:00:00Z") };
await copying.set("task.1", task2);
task2.title = "CHANGED BY ACCIDENT";
const copy = (await copying.get<typeof task2>("task.1")).value;
console.log(copy?.title, copy?.due instanceof Date);

CHANGED BY ACCIDENT
Buy milk true

With a serializer, the cache turns the value into text on the way in and builds a new object on the way out, so each reader gets its own copy. JsonCacheSerializer also keeps Date objects as dates, which plain JSON does not. You will see why in the serialization lesson. A cache in another process, such as Redis, always stores text, so using a serializer from the start also makes the memory adapter behave like the real thing.

Locks: one at a time

A lock is a named ticket that only one piece of code can hold at a time. Anyone else who asks for the same name waits, or gives up. Use it around work that must not run twice at once, such as rebuilding a daily report.

withLock(name, fn) takes the lock, runs fn, and releases the lock, even if fn throws:

import { createCacheService, createMemoryCacheAdapter, isCacheError } from "@zudojs/cache";

const wait = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));
const cache = createCacheService({ adapter: createMemoryCacheAdapter() });

async function rebuildReport(who: string, retryAttempts = 3): Promise<string> {
  return cache.withLock("daily-report", async () => {
    console.log(who, "starts");
    await wait(50);
    console.log(who, "ends");
    return who;
  }, { ttl: 10_000, retryAttempts });
}

console.log(await Promise.all([rebuildReport("A"), rebuildReport("B")]));

const patient = rebuildReport("C");
const impatient = rebuildReport("D", 0).catch((error: unknown) => {
  if (isCacheError(error)) console.log("D gave up:", error.code);
});
await Promise.all([patient, impatient]);

A starts
A ends
B starts
B ends
[ 'A', 'B' ]
C starts
D gave up: CACHE_LOCK_UNAVAILABLE
C ends

A and B asked at the same time. B had to wait: it tried again (by default up to 3 times, 100 ms apart) and ran once A released the lock. They never overlapped. D asked with retryAttempts: 0, found the lock taken by C, and gave up straight away with CACHE_LOCK_UNAVAILABLE, before C had even finished.

The ttl makes the lock a lease: if your process crashes while holding it, the lock frees itself after 10 seconds instead of blocking everyone forever. While fn runs, the lease is renewed for you, and if it is lost anyway, withLock throws instead of pretending it worked.

See what the cache is doing

A cache you cannot measure is a guess. The most important number is the hit rate: hits divided by all reads. A hit rate of 0.9 means 9 of 10 reads were served from the cache. If it is low, your TTL may be too short, or your keys too specific to ever repeat.

The service counts hits, misses, sets, deletes and errors, and emits an event on every operation that you can listen to:

import { createCacheService, createMemoryCacheAdapter } from "@zudojs/cache";

const cache = createCacheService({ adapter: createMemoryCacheAdapter() });

const subscription = cache.subscribe("cache.miss", (event) => {
  console.log("miss:", event.key);
});

await cache.get("tasks.list", { namespace: "ada" });
await cache.set("tasks.list", ["Buy milk"], { namespace: "ada" });
await cache.get("tasks.list", { namespace: "ada" });
await cache.get("tasks.list", { namespace: "ada" });
await cache.get("tasks.list", { namespace: "linus" });
subscription.unsubscribe();

console.log(cache.getStats());
console.log(cache.getHotKeys(3));
console.log("entries:", await cache.size());

miss: zudojs:ada:tasks.list
miss: zudojs:linus:tasks.list
{ hits: 2, misses: 2, sets: 1, deletes: 0, errors: 0, hitRate: 0.5 }
[ { key: 'zudojs:ada:tasks.list', hits: 2 } ]
entries: 1

• subscribe(type, handler) listens for cache.hit, cache.miss, cache.set, cache.delete, cache.clear or cache.error, or "*" for all. You will learn how events work in general in the next lesson.
• errors counts failed operations, including input the cache refused, such as a bad key, namespace, pattern or tag. Each one also emits cache.error. A number that keeps growing usually means a bug in how you build keys.
• getHotKeys(n) lists the most-read keys. It tells you where caching pays off most.
• getLatencyStats(CacheOperation.GET) gives you how long reads take (p50, p95, p99). You will send numbers like these to a dashboard in the observability lesson.

Put it together: a cached task service

Here is the Task API's service with caching built in. Reads go through getOrSet, scoped to the user and tagged. Writes go to the database first, then clear the user's task entries:

import { createCacheService, createMemoryCacheAdapter, JsonCacheSerializer } from "@zudojs/cache";
import type { CacheService } from "@zudojs/cache";
import type { Task, TaskDb } from "./task-db.js";

export function createTaskCache(): CacheService {
  return createCacheService({
    adapter: createMemoryCacheAdapter({ maxEntries: 10_000 }),
    config: { prefix: "taskapi", defaultTtl: 5 * 60_000, serializer: new JsonCacheSerializer() },
  });
}

export class CachedTaskService {
  constructor(
    private readonly db: TaskDb,
    private readonly cache: CacheService,
  ) {}

  async list(userId: string): Promise<Task[]> {
    const result = await this.cache.getOrSet("tasks.list", () => this.db.listTasks(userId), {
      namespace: userId,
      tags: ["tasks"],
    });
    return result.value;
  }

  async create(userId: string, title: string): Promise<Task> {
    const task = await this.db.addTask(userId, title);
    await this.cache.invalidateByTag(["tasks"], { namespace: userId });
    return task;
  }
}

import { CachedTaskService, createTaskCache } from "./cached-task.service.js";
import { TaskDb } from "./task-db.js";

const db = new TaskDb();
const cache = createTaskCache();
const tasks = new CachedTaskService(db, cache);

console.log("ada:", (await tasks.list("ada")).map((t) => t.title));
console.log("ada:", (await tasks.list("ada")).map((t) => t.title));
console.log("linus:", (await tasks.list("linus")).map((t) => t.title));

await tasks.create("ada", "Call the bank");
console.log("ada:", (await tasks.list("ada")).map((t) => t.title));

console.log("queries:", db.queries);
console.log("hit rate:", cache.getStats()?.hitRate);

ada: [ 'Buy milk', 'Write report' ]
ada: [ 'Buy milk', 'Write report' ]
linus: [ 'Fix bug' ]
ada: [ 'Buy milk', 'Write report', 'Call the bank' ]
queries: 4
hit rate: 0.25

Follow the four queries: Ada's first list, Linus's list, the insert, and Ada's list again after create cleared it. Ada's second read was a hit, so the hit rate is 1 of 4 reads. The new task shows up straight away because create cleared the stale list, and Linus's cache was never touched.

In the Task API, create the cache once when the app starts and register it in the container, as you did with other services in the dependency injection lesson, so every request shares the same cache.

Practice

import { createCacheService, createMemoryCacheAdapter } from "@zudojs/cache";

const cache = createCacheService({ adapter: createMemoryCacheAdapter() });
let queries = 0;

async function loadTask(id: number) {
  queries += 1;
  return { id, title: "Buy milk" };
}

async function get(userId: string, id: number) {
  const result = await cache.getOrSet(`task.${id}`, () => loadTask(id), {
    namespace: userId,
    tags: ["tasks"],
  });
  return result.value;
}

await get("ada", 1);
await get("ada", 1);
console.log("queries:", queries);

await cache.invalidateByTag(["tasks"], { namespace: "ada" });
await get("ada", 1);
console.log("queries:", queries);

queries: 1
queries: 2

import { createCacheService, createMemoryCacheAdapter } from "@zudojs/cache";

const wait = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));
const cache = createCacheService({ adapter: createMemoryCacheAdapter() });

for (let i = 0; i < 10; i++) {
  await cache.getOrSet("tasks.list", async () => ["Buy milk"], { ttl: 30 });
  await wait(10);
}
const stats = cache.getStats();
console.log("misses:", stats?.misses, "hits:", stats?.hits);

misses: 4 hits: 6

Recap

• A cache keeps copies of slow results. A hit reads the copy, a miss does the work. The price is stale data.
• get returns { hit, value }. getOrSet does read, compute and store in one call, and stops a stampede of identical queries.
• Keys are prefix:namespace:key, with no colons in your parts. The key or namespace must include everything the answer depends on, especially the user.
• A TTL limits how long a copy can be wrong. ttl: null never expires, and 0 is not allowed.
• Every write path invalidates the entries it made stale: by key, by tag, or by pattern, in the same namespace.
• withLock runs work one at a time. The built-in lock and memory adapter only cover one process.
• Watch the hit rate with getStats() and events, and use failSilently so a broken cache is only slow, not fatal.