[harlan-zw]

🔗 Linked issue

Related to #540

❓ Type of change

• 📖 Documentation
• 🐞 Bug fix
• 👌 Enhancement
• ✨ New feature
• 🧹 Chore
• ⚠️ Breaking change

📚 Description

Every cached route had the bug that #540 fixed for App mints. Nitro's defineCachedFunction keeps one in-flight promise per key for the whole isolate, and singleFlight in shared/server/cache.ts did the same. The workerd runtime ties I/O to the request that started it. If that request answers before its resolve finishes, workerd drops the I/O and the promise never settles. Every later read of that key in the isolate then waits on it forever.

What changes:

• patches/nitropack@2.13.4.patch keeps Nitro's in-flight resolves per H3 event. Reads inside one request still share one resolve. A call without an event resolves alone. This covers 14 defineCachedEventHandler routes, the tag profile defineCachedFunction, and llms.txt from nuxt-ai-ready.
• shared/server/cache.ts drops its isolate-wide promise map. A cold or dead key computes in each request that reads it. This covers 12 cached() and readThroughCache() call sites.
• A stale key still refreshes once per isolate. The first stale read takes a 30 second refresh claim, and other stale reads serve the stale value meanwhile. The claim is an expiry time, so a request that ends early cannot strand it. It lapses, and the next stale read refreshes.
• readThroughCache now serves the stale value while another request recomputes it. Before, it waited for that recompute.

What this costs:

• Concurrent cold misses on one key each compute, as concurrent cold mints did after fix(github): keep each App mint inside the request that started it #540. Three tests that pinned one compute per cold burst are removed.
• Concurrent stale reads of a Nitro SWR route each refresh. The patch leaves the rest of Nitro's cache logic unchanged.

🤖 AI disclosure: Harlan Agent Kit modified this description. My AI open-source policy.

[harlan-zw]

Agent note from Harlan Agent Kit: checked by hand in local workerd, since CI cannot cover it.

Setup: pnpm build, then wrangler dev on .output/server/wrangler.json. A temporary route, removed before commit, called each cache with a resolver that fetched a local upstream answering after 1.5 s. Request A answered at 300 ms and left its resolve running. Request B read the same key at 500 ms, and request C at 3.5 s. The client gave up after 45 s.

Case	Before	After
Nitro defineCachedFunction, cold key	B and C hung past 45 s; the resolve never settled	B 1.55 s, C 1.57 s
cached(), cold key	B and C hung past 45 s	B 1.56 s, C 44 ms (stale entry)
defineCachedEventHandler with swr, stale key	Refresh ran under waitUntil and finished	Same; B started its own refresh
A's client aborts at 300 ms	Local workerd kept A running, so B got A's value at 1.06 s	Not reproducible locally

The upstream log showed workerd closing A's fetch right after A answered.

One case stays open. A defineCachedFunction with swr and no event argument has no waitUntil for its refresh. Before, the first refresh was cut off and the key stayed stale for the isolate's life. After, each stale read starts a refresh, and each is still cut off. No call site here does this: the tag profile and llms.txt pass the event first.