Firefox does not resolve fetch() on held POST /run until a body byte #321

Summary

POST /service/control/run holds the HTTP connection until the worker finishes (often 10–60s) and only then writes JSON via c.JSON. Chromium and WebKit treat flushed response headers as enough for fetch() to resolve. Firefox does not: fetch() stays pending until it sees at least one body byte (nsUnknownDecoder / no OnDataAvailable).

This is independent of upgrading Playwright. It still happens on current and newer Firefox builds.

Related: #315 (streaming preamble workaround), #318 (Playwright 1.62 upgrade; CI also saw flaky waitForResponse("**/service/control/run") on Firefox), #320 (live logs / job API, which would avoid a held POST).

What the product does today

1. Browser fetch("/service/control/run", { method: "POST", body: JSON }) in frontend/src/utils.ts (runCode).
2. Control publishes one AMQP job to a one-shot worker pod and blocks the HTTP handler until Subscribe() gets the single RPC reply (control-service/main.go handleRun).
3. Then c.JSON(...) with the full WorkerResponsePayload.
4. Caddy reverse_proxy /service/control/* control:8080 with default buffering.
5. CI hits the stack over HTTP/1.1 (http://127.0.0.1:nodePort), not HTTP/2/3.

The UI does not stream logs; it waits for the complete JSON. The Firefox bug is not “we need headers early for UX”. It is that if headers get flushed without a body (Echo/Caddy/chunked encoding), Firefox’s fetch() never resolves until a body byte (or EOS). A quiet stream can also surface as TypeError: NetworkError when attempting to fetch resource, then AbortError.

Upstream

• Mozilla 1544313 — fetch/response body stream buffers when sniffing; fetch() does not resolve until ~512 bytes or EOS. Fixed in Firefox 150 for the missing-Content-Type case only.
• 1759996 — duplicate; NetworkError on a quiet chunked stream. Workarounds mentioned: Content-Type and/or padding newlines.
• 1793342 — same decoder on HTTP/3; even with Content-Type, some people needed { cache: "no-store" }.

The case that bites us is headers flushed + Content-Type: application/json + empty body until the worker is done. That is not the missing-Content-Type sniffing bug that landed in 150. Empty body ⇒ no OnDataAvailable ⇒ fetch() stays pending.

Playwright’s own library tests still comment: “In Firefox, |fetch| will be hanging until it receives |Content-Type| header from server” (page-network-response.spec.ts). They always write a body byte (hello ) after setting Content-Type.

Repro (no Caddy, HTTP/1.1)

Node server: Content-Type: application/json, writeHead(200), flushHeaders(), wait 1500ms, then end(JSON). Client in the page: fetch(url, { method: "POST", cache: "no-store" }). Measure time until fetch() resolves (not until resp.text()).

Optional mode=dummy: write "\n" immediately after flush.

Results (2026-09-03)

Same numbers for XMLHttpRequest: readyState === 2 (HEADERS_RECEIVED) also waits for the first body byte on Firefox. Switching runCode from fetch to XHR does not help.

cache: "no-store" was already on in the repro and did not unblock Firefox.

Control case hold-all (nothing written until JSON is ready, like today’s c.JSON at the end): both fetch and XHR wait ~1500ms and then succeed. That is correct. The failure mode is headers already on the wire, body still empty.

What does not fix it

• Upgrading Playwright / Firefox (still broken through Firefox 155).
• fetch(..., { cache: "no-store" }).
• Setting request Content-Type (already set).
• Replacing fetch with XMLHttpRequest.
• Waiting for Mozilla 1544313 “fixed in 150” — that was missing Content-Type sniffing only.

Workarounds that do work

Smallest server fix (what #315 does)

After Publish:

1. Set Content-Type: application/json, 200, write a JSON-whitespace "\n", Flush. That is the byte Firefox needs.
2. Optional: "\n" every 1s while the worker runs (idle insurance for a 60s wait; not required to unblock fetch() once the preamble ran).
3. Encode the real object afterward (json.Encoder / leading whitespace is valid JSON).
4. Caddy: flush_interval -1 on reverse_proxy /service/control/* plus timeouts matching EXECUTION_TIMEOUT, so the proxy does not hold the first byte.

Heartbeat without Caddy flush can still leave Firefox stuck if Caddy buffers the preamble.

Better product shape (do not hold POST)

The UI is fire-and-forget: one result blob (logs, files, duration). Professional HTTP APIs for 10–60s jobs usually:

1. Job + poll (best fit for current UX)
   POST /runs → 202 { id } as soon as the job is queued.
   GET /runs/:id until succeeded|failed.
   Each response is a small complete JSON document. Firefox, Caddy, Cloudflare, and retries all behave.

2. SSE if we want live logs (Live logs: stream worker stdout over a job API (consider Redis later) #320)
   Same POST /runs, then GET /runs/:id/events with native EventSource (GET-only). Send : connected\n\n immediately (idiomatic SSE, also the first-byte lesson). Do not implement SSE as fetch(POST) + response.body — that is the same Firefox path.

3. WebSocket only if we need a session (cancel, stdin, REPL). Overkill for Run as it works today.

Holding /run and papering over Firefox with a preamble newline is a valid hotfix, not the long-term API.

CI symptom

On #318 e2e, Firefox (and sometimes Chromium) tests flaked with:

page.waitForResponse: Test timeout of 120000ms exceeded.
waiting for response "**/service/control/run"

Retries often passed. That matches a client fetch() that never sees a complete response (or sits until worker timeout) when headers were flushed without a body, or the connection went idle. Separate from the Firefox 153 Monaco ts.worker tab crash on the share test (fixed on #318).

Suggested next step

• Short term: land a minimal version of Keep /run streaming so Firefox fetch() sees a first byte #315 if we need CI/Firefox stability before a job API — preamble \n + Caddy flush_interval -1; add heartbeat only if idle NetworkErrors remain.
• Medium term: job + poll (or SSE as in Live logs: stream worker stdout over a job API (consider Redis later) #320) so /run is not a 60s held POST.