Skip to content

Workers Cache stores an older in-flight response after publish/unpublish purge (stock Astro provider) #7732

Open

Description

@coreylight

Summary and requested confirmation

On the live Cloudflare Workers platform, a response rendered before an EmDash publish/unpublish can finish after invalidation has taken effect and then be stored as fresh Workers Cache HTML. Subsequent canonical requests are HTTP200/HIT with the old content, including content just unpublished.

This reproduces with the unmodified installed Astro Cloudflare cache provider and a minimal direct EmDash page/handler, as well as with our application's response wrapper/layout. Ordinary warm-cache invalidation passes. An independently warmed alternate URL for the same database entry ID changes to current MISS/private404 during the overlapping write, before its TTL expires. We are not inferring purge execution merely from an HTTP-successful CMS write.

Could the Workers Cache team please confirm:

  1. Is cache.purge() intended to prevent an older, already-running response from populating the cache after the purge takes effect, or does it only remove entries that exist at purge time?
  2. Does the answer differ for tag, path-prefix, purgeEverything or soft cache.invalidate() operations, streaming responses, or collapsed requests?
  3. If our observed behavior is unintended, is there a supported platform fix, rollout/compatibility requirement or known tracking issue? If it is expected, what supported architecture prevents post-write stale fills without a bespoke coordination store or a database read on every cache hit?
  4. What precisely does success: true guarantee: request acceptance, global propagation, or a barrier against older in-flight fills? What freshness contract should a CMS advertise?

I am filing in workerd because this concerns the runtime cloudflare:workers cache API and the same per-entrypoint platform surface discussed in #6865. I have not established a defect in open-source workerd itself. This requires the production Workers Cache embedder/edge cache; please route to the owning platform team or correct intake if necessary. Plain local workerd is not a substitute for this reproduction.

Tested environment

  • Live staging-only Cloudflare Worker at its canonical workers.dev ingress; no production mutations.
  • Astro 7.2.8; @astrojs/cloudflare 14.2.4; Wrangler 4.131.0.
  • EmDash 1.1.0, using our site's reviewed vendored package. The isolation trial did not replace that package with a pristine upstream tarball; it did select the unmodified cacheCloudflare() provider and direct public query/handler APIs.
  • D1-backed content; no EmDash KV object cache configured.
  • Native version readback confirms compatibility date 2026-02-24, flags nodejs_compat, no placement configured, cross-version cache disabled, default gateway cache disabled and named CachedMarketingSite cache enabled.
  • Edge lifetime 60 seconds; swr: 0.
  • Test date: 2026-10-13; actual gateway request.cf.colo DUB for completed controls/overlap/freshness observations.
  • Real HTTP I/O: Node 24.21.0, installed Undici 7.29.1 Agent, HTTP/1.1, two persistent connections, IPv4, 120-second keepalive. No injected I/O doubles in the live report.
  • Live trial native version ca6ed36b-2f15-4c3c-80cc-b360dff45179, deployment 0a42f242-bc20-453f-9841-ef515574c79d (100%). Account/resource identifiers, credentials, sessions, signed preview URLs and private CMS content are excluded from this public report.

Minimal reference path

The following is a small extraction of the tested reference path for an existing staging EmDash/Astro site. These are the handler/query/hold mechanisms used in the live trial, with application guards and unrelated routes omitted for clarity; this is not a claim that a separate framework-free Worker was tested. Use only owned synthetic staging content.

Configuration

Select the stock provider and official route rules:

import { cacheCloudflare } from "@astrojs/cloudflare/cache";
// Keep the existing Cloudflare adapter, EmDash D1 integration and output: "server".
cache: { provider: cacheCloudflare() },
routeRules: { "/cache-reference-595/**": { maxAge: 60, swr: 0 } }

The deployed Wrangler cache fragment was:

{
  "cache": { "enabled": false },
  "exports": {
    "default": { "type": "worker", "cache": { "enabled": false } },
    "CachedMarketingSite": { "type": "worker", "cache": { "enabled": true } }
  }
}

The uncached gateway calls the cached named entrypoint's HTTP fetch() for anonymous public reads, and its custom RPC method for CMS writes and private traffic, keeping EmDash's invalidation in the cache-owning entrypoint. This follows the documented gateway pattern: https://developers.cloudflare.com/workers/cache/examples/ .

Minimal Astro page: src/pages/cache-reference-595/[nonce].astro

---
import { LiveEntryNotFoundError } from "astro/content/runtime";
import { getEmDashEntry } from "emdash";
const nonce = Astro.params.nonce ?? "";
if (!/^[a-f0-9]{8}$/.test(nonce)) {
  return new Response("Not found", { status: 404,
    headers: { "Cache-Control": "private, no-store" } });
}
const { entry, error, cacheHint } = await getEmDashEntry(
  "newsroom", `staging-cache-595-${nonce}`
);
if (Astro.cache.enabled) Astro.cache.set(cacheHint);
if (error && !LiveEntryNotFoundError.is(error)) throw new Error("Reference query unavailable");
if (!entry) return new Response("Not found", { status: 404,
  headers: { "Cache-Control": "private, no-store" } });
---
<html lang="en"><head><title>Owned cache reference</title>
<meta name="robots" content="noindex,nofollow" /></head>
<body><h1>{entry.data.title}</h1></body></html>

No application layout, navigation query, custom middleware cache defaults, custom cache key, provider wrapper, response-policy wrapper or validator rewriting is used on this reference path. Native EmDash middleware still runs. An outer uncached gateway adds observation headers and protects private/disabled traffic.

Direct handler and owned streaming hold

Inside the existing named cache-owning entrypoint, the reference path calls the official handler directly:

import emdashWorker from "@emdash-cms/cloudflare/worker";
import { WorkerEntrypoint } from "cloudflare:workers";

async function holdOwnedFill(response) {
  if (response.status !== 200) return response;
  const body = await response.text(); // Only the tiny owned test fixture is buffered.
  const headers = new Headers(response.headers);
  headers.delete("Content-Length");
  headers.set("X-Rendered-At", new Date().toISOString());
  const encoder = new TextEncoder();
  return new Response(new ReadableStream({ async start(controller) {
    controller.enqueue(encoder.encode("<!-- owned streaming race probe -->"));
    await new Promise(resolve => setTimeout(resolve, 45000));
    controller.enqueue(encoder.encode(body));
    controller.close();
  }}), { status: response.status, headers });
}

export class CachedMarketingSite extends WorkerEntrypoint {
  async fetch(request) { return this.uncached(request); }
  async uncached(request) {
    const native = await emdashWorker.fetch(request, this.env, this.ctx);
    const isOwned = request.method === "GET" &&
      /^\/cache-reference-595\/[a-f0-9]{8}\/$/.test(new URL(request.url).pathname);
    return isOwned && request.headers.get("X-Cache-QA") === "hold"
      ? holdOwnedFill(native) : native;
  }
}

export default {
  ...emdashWorker,
  async fetch(request, env, ctx) {
    const url = new URL(request.url);
    const isReferenceRead = request.method === "GET" &&
      /^\/cache-reference-595\/[a-f0-9]{8}\/$/.test(url.pathname) &&
      !url.search && !request.headers.has("Cookie") &&
      !request.headers.has("Authorization") && !request.headers.has("Range");
    // No custom cf.cacheKey for the reference path.
    return isReferenceRead
      ? ctx.exports.CachedMarketingSite.fetch(request)
      : ctx.exports.CachedMarketingSite.uncached(request);
  }
};

The shortened snippet names the hold header X-Cache-QA; the actual guarded trial used X-Marketing-Cache-QA, restricted it to owned staging fixtures, and confirmed reference selection with a separate response marker. The hold buffers already-rendered old HTML, records server capture time, emits an initial comment, and delays the remaining body by 45 seconds. It does not change the database or affect ordinary page traffic.

Reproduction sequence and causal controls

Create an owned newsroom entry with slug staging-cache-595-<eight hex digits> and a synthetic title. Use authenticated EmDash CMS writes in the same named entrypoint. Installed EmDash single-entry hints tag the database entry ID, not automatically the collection name.

  1. Publish title OLD. Warm the exact reference URL until native HIT.
  2. Publish title CONTROL. Require CONTROL to appear at an edge with a verified pre-write HIT before 60-second expiration. This ordinary invalidation control passed in every completed case.
  3. Publish OLD again, which invalidates the reference URL. Warm the existing normal article URL for the same entry until HIT. This alternate URL is the independent purge control; a different entry in the collection would be an invalid control.
  4. Start the reference GET with the owned hold header and wait approximately 10 seconds. The server has captured OLD but has not completed its body.
  5. Either update/publish title NEW, or unpublish the entry. Record mutation start/end. Do not retry a committed write blindly.
  6. Read the independently warmed alternate URL. Require current NEW/HTTP200/MISS, or absent OLD/HTTP404/BYPASS, at a previously warmed gateway edge and before expiry. This passed during each overlapping write, proving actual invalidation took effect.
  7. Await the held response; assert capture < mutation start <= mutation end < capture + 45s and confirm the held body is OLD. This server-side oracle passed in all four completed cases.
  8. For the original 15-second post-fill budget, poll the exact canonical held URL without cache-busting queries or no-cache headers. Publication should show NEW; unpublication should return404 without OLD. Instead it returned24 old HTTP200/HIT samples in each case.
  9. Unpublish owned fixtures and read back exact IDs/slugs as drafts. All four cleanup readbacks passed; nothing was deleted.

The final runner warmed both persistent connections and recorded a per-colo map of actual pre-write HITs. Inbound request.cf.colo was captured before forwarding and stamped by the uncached gateway after lookup; forwarded colo and Cf-Ray suffix were stored separately. Cached origin location headers were not trusted. Earlier attempts that stopped at a location, transport or incorrectly separate-entry purge control are inconclusive and are not counted as freshness failures.

Staging evidence

All timestamps below are UTC on 2026-10-13. Each case passed ordinary warm invalidation, independent same-entry alternate-URL invalidation, server overlap and draft cleanup. Each subsequent freshness series contained 24 HTTP200/HIT old-body observations, with actual gateway colo DUB.

Handler Write Old render captured Mutation started Mutation finished Expected after held fill Observed
Minimal reference Publish 17:16:20.306 17:16:29.242 17:16:50.865 NEW/200 OLD/200/HIT
Minimal reference Unpublish 17:18:41.484 17:18:50.618 17:18:59.072 absent/404 OLD/200/HIT
Application wrapper/layout Publish 17:21:02.050 17:21:09.076 17:21:31.471 NEW/200 OLD/200/HIT
Application wrapper/layout Unpublish 17:23:31.371 17:23:38.335 17:23:46.727 absent/404 OLD/200/HIT

Reference publication's alternate URL was current HTTP200/MISS; reference removal's alternate URL was HTTP404/BYPASS without OLD. Both were at DUB with verified pre-write HITs and before TTL expiration. The same control passed for the wrapped cases. This rules out our provider/response wrapper, layout and custom key as necessary causes in this comparison, not every possible framework interaction.

Earlier application-path tests also reproduced overlap failure with checked native purgeEverything hard purging and native soft invalidation under zero-SWR/no-stale policies. Those are separate trials; the four-case stock-provider comparison above uses the installed provider's tag purging. Native purge failure handling and ordinary publish/update/unpublish privacy tests pass; neither substitutes for the overlap oracle.

The 45-second hold deliberately widens the race. It is not a measurement of normal render duration, real-world race frequency, or a production memory/capacity claim. The 15-second observation budget is shorter than TTL; it is our acceptance threshold, not a claimed Cloudflare contractual SLA. We did not weaken TTL or freshness assertions to declare success.

Mitigation and related work

We restored hardened native purge-result/error handling, removed the diagnostic reference route/policy bypass, and left staging runtime page caching disabled. Production caching was not enabled or deployed by this work. All owned test records are drafts. The published staging report is historical evidence, not a request to mutate our live Worker or grant access to our account.

Related but different issues/fixes:

Please confirm the supported purge/fill ordering contract and the appropriate fix or supported workaround. If no in-flight barrier is provided, an explicit documented limitation would let CMS users choose an honest bounded-staleness policy instead of assuming publish/unpublish freshness that the platform does not guarantee.

Activity

  1. coreylight commented on Oct 13, 2026

    @coreylight
    Author

    @danlapid — you helped route the native Workers Cache/entrypoint issue in #6865. Could you route this purge-versus-in-flight-fill question to the Workers Cache team? The report includes a minimal EmDash reference path and four completed live-staging cases with independent warm invalidation controls. We need confirmation of the intended ordering contract and a supported fix/workaround, rather than assuming that purge acceptance fences older renders. No production change is being requested.

  2. zaidoon1 commented on Oct 14, 2026

    @zaidoon1
    Contributor

    Is cache.purge() intended to prevent an older, already-running response from populating the cache after the purge takes effect, or does it only remove entries that exist at purge time?

    It only acts on what's already cached when the purge is processed. It isn't a barrier against fills that are still in flight.

    If your Worker started a response before your write and is still generating or streaming it when the purge lands, the purge doesn't touch it. When that response completes, it's stored like any other fill, old content included. Purge is fast, so a slow response, like the held one in your test, easily outlasts it.

    Does the answer differ for tag, path-prefix, purgeEverything or soft cache.invalidate() operations, streaming responses, or collapsed requests?

    No. Tag, path prefix and purgeEverything purges and soft invalidation all act only on what's already cached, so none of them stop an in-flight fill from being stored. Streaming responses behave the same way, and collapsed requests that join an in-flight fill get that same old response. Whether a given race gets caught comes down to timing, which is why it can look like it works some of the time.

    If it is expected, what supported architecture prevents post-write stale fills without a bespoke coordination store or a database read on every cache hit?

    Add a delay before sending the purge. Make the delay longer than your slowest response takes to render and finish streaming, plus any time your write needs to reach every read path (read replicas, for example). A Queue message with delaySeconds works well for this. Purges are scoped to the calling entrypoint, so have the queue consumer call an RPC method on CachedMarketingSite that does the purge.

    What precisely does success: true guarantee: request acceptance, global propagation, or a barrier against older in-flight fills?

    Request acceptance only. Purge is asynchronous: success: true means we've received and queued your purge and won't lose it. It doesn't mean the purge has propagated, and it doesn't hold back in-flight fills.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions