Skip to content

Miniflare resets the connection when a Worker responds without reading the request body; workerd does this whenever the body crossed a service binding #7634

Open

Description

@michael-huffaker

What versions & operating system are you using?

  • Windows 11 (10.0.26200), Node 24.14.1. Measured on this machine only.
  • miniflare 5.20260921.0-alpha (the latest tag) with its workerd 1.20260921.1, and that workerd run on its own.
  • The newest workerd, 1.20260923.1, gives the same results, on its own and under the same Miniflare (through an npm overrides entry).
  • Where we first saw it: @cloudflare/vite-plugin 1.57.3 and wrangler 4.136.3, with Miniflare's undici 7.29.0.

Please provide a link to a minimal reproduction

Inline below: npm i miniflare, then node repro.mjs. It runs Miniflare, and it runs the workerd that Miniflare installs directly, without Miniflare.

Describe the Bug

When a Worker responds without reading the request body, and the body reached it through a service binding, workerd resets the client's connection a few milliseconds after the response. Miniflare's entry Worker passes each request to the user's Worker through a service binding (await service.fetch(request) in packages/miniflare/src/workers/core/entry.worker.ts), so under Miniflare this happens to any Worker that answers early - for example one that refuses an oversized upload with 413 before reading it. A client that has not yet read the response when the reset arrives loses it: all 20 did when the client started reading 300 ms after sending.

Without the binding, workerd keeps the connection, as Node's HTTP server does. That fits KJ's HttpServer: when a service responds without reading the whole body, it reads and discards up to canceledUploadGraceBytes (64 KiB) within canceledUploadGracePeriod (1 s) before deciding whether to close (HttpServerSettings in kj/compat/http.h). Through a binding, a 5,000-byte body, well inside that grace, is reset as if there were none.

Since workerd alone reproduces it, the fault looks to be in workerd rather than Miniflare. We are filing it here because Miniflare is where it surfaces; it may belong in cloudflare/workerd.

From repro.mjs below, with workerd 1.20260921.1. Each request sends its headers and its whole body in one write, and every server answers 413. Every row is 20 requests with the same outcome. "Kept" means the connection was still open 2 s later, past the 1 s grace; "reset" means the client's socket failed with ECONNRESET.

Server Body Client reads at once Client reads 300 ms later
workerd: Worker does not read the body 5,000 bytes 413, kept 413, kept
workerd: Worker forwards through a service binding to that Worker 5,000 bytes 413, then reset 3.3-4.2 ms later no response, reset
Miniflare: Worker does not read the body 5,000 bytes 413, then reset 3.2-33.9 ms later no response, reset
workerd, control: Worker reads the body 5,000 bytes 413, kept 413, kept
workerd, control: forwards through a service binding to a Worker that reads it 5,000 bytes 413, kept 413, kept
Miniflare, control: Worker reads the body 5,000 bytes 413, kept 413, kept
Node http server, reference: does not read the body 5,000 bytes 413, kept 413, kept
workerd: Worker does not read the body 100,000 bytes, past the grace 413, then reset 0.1-0.2 ms later no response, reset

RFC 9112 §9.6 is why a reset matters: "If a server performs an immediate close of a TCP connection, there is a significant risk that the client will not be able to read the last HTTP response."

How it shows up. We found it as a rare 500 from vite preview, on a POST that a Worker refuses with 413 before reading the body: the Vite plugin's dispatchFetch rejected with TypeError: fetch failed (stack below) and the plugin answered 500. In runs of 300 and 1,000 requests through mf.dispatchFetch, with a streamed 5,011-byte body and 16 threads keeping every core busy, one request in each failed that way, with read ECONNRESET as the cause. On an idle machine all 1,000 succeeded, and so did a later 8,000 with a 5,000-byte body under the same or heavier load. It takes the client being slow to read at the wrong moment; repro.mjs controls when the client reads, so it reproduces every time.

Related. cloudflare/workers-sdk#15709 looks like a relative: a Worker cancels a ~2 MB body part-way under unstable_dev, and the connection is lost. The last row above shows that even served directly, workerd resets once more than the 64 KiB grace is left unread: that is KJ's grace running out, and the staged close below would address it. This report is the case inside the grace, which fails only because the body crossed a binding - and in Miniflare every body has.

What this does not establish.

  • Measured on Windows 11 only. Whether a reset erases a response the client has already received depends on the client's TCP stack; on Windows it does (the 300 ms column).
  • We haven't traced where the drain is lost after a binding. The reset follows the response by a few milliseconds, far sooner than the 1 s grace, so it looks as if the drain is not attempted, or fails at once.
  • Not tried against deployed Workers.

Suggested fix

  • Give an unread body that crossed a service binding the same grace as one that did not: drain it up to canceledUploadGraceBytes within canceledUploadGracePeriod, then keep the connection or close it cleanly.
  • Past the grace, the staged close RFC 9112 §9.6 describes would stop the reset from erasing the response already sent: "First, the server performs a half-close by closing only the write side of the read/write connection. The server then continues to read from the connection until it receives a corresponding close by the client, or until the server is reasonably certain that its own TCP stack has received the client's acknowledgement of the packet(s) containing the server's last response. Finally, the server fully closes the connection." That would cover [Bug]: unstable_dev PUT after oversized-body cancellation returns 500 "Network connection lost" workers-sdk#15709 as well.

Please provide any relevant error logs

The error behind the 500, as vite preview printed it (paths shortened, frames elided):

TypeError: fetch failed
    at Object.processResponse (...node_modules/undici/lib/web/fetch/index.js:237:16)
    ...
    at async fetch4 (...node_modules/miniflare/dist/src/index.js:64836:20)
    at async _Miniflare.dispatchFetch (...node_modules/miniflare/dist/src/index.js:119777:22)
    at async file:///.../node_modules/@cloudflare/vite-plugin/dist/index.mjs:51227:19
repro.mjs
// A request body that crosses a service binding and is not read gets the connection reset.
// npm i miniflare (which brings workerd), then node repro.mjs. Runs workerd on its own with four
// Workers, Miniflare with one, and a Node HTTP server as a reference. Every request goes over a raw
// socket and declares and sends its body with the headers in one write; every server answers 413.
import { spawn } from "node:child_process";
import { mkdtempSync, rmSync, writeFileSync } from "node:fs";
import http from "node:http";
import net from "node:net";
import os from "node:os";
import path from "node:path";
import { convertV4MiniflareOptions, Miniflare } from "miniflare";
import workerd from "workerd";

const REFUSE = `export default { async fetch() { return new Response("too large", { status: 413 }); } };`;
const READ = `export default { async fetch(request) { await request.arrayBuffer(); return new Response("too large", { status: 413 }); } };`;
const FORWARD = `export default { async fetch(request, env) { return env.TARGET.fetch(request); } };`;
const worker = (script, target) =>
  `(modules = [(name = "worker.js", esModule = ${JSON.stringify(script)})], compatibilityDate = "2026-09-01"` +
  (target ? `, bindings = [(name = "TARGET", service = "${target}")])` : ")");

const PORT = { refuse: 18701, read: 18702, forwardToRefuse: 18703, forwardToRead: 18704, node: 18705 };
const config = `using Workerd = import "/workerd/workerd.capnp";
const config :Workerd.Config = (
  services = [
    (name = "refuse", worker = ${worker(REFUSE)}),
    (name = "read", worker = ${worker(READ)}),
    (name = "forwardToRefuse", worker = ${worker(FORWARD, "refuse")}),
    (name = "forwardToRead", worker = ${worker(FORWARD, "read")}),
  ],
  sockets = [
    (name = "refuse", address = "127.0.0.1:${PORT.refuse}", http = (), service = "refuse"),
    (name = "read", address = "127.0.0.1:${PORT.read}", http = (), service = "read"),
    (name = "forwardToRefuse", address = "127.0.0.1:${PORT.forwardToRefuse}", http = (), service = "forwardToRefuse"),
    (name = "forwardToRead", address = "127.0.0.1:${PORT.forwardToRead}", http = (), service = "forwardToRead"),
  ],
);
`;

// One request with a body of `bytes`. The client starts reading the response `readAfterMs` after
// sending, as a busy client would.
function probe(port, bytes, readAfterMs) {
  return new Promise((resolve) => {
    const socket = net.connect(port, "127.0.0.1");
    const chunks = [];
    let respondedAt;
    let settled = false;
    const finish = (ending) => {
      if (settled) return;
      settled = true;
      const status = /^HTTP\/1\.1 (\d{3})/.exec(Buffer.concat(chunks).toString("latin1"))?.[1];
      const after = respondedAt === undefined ? undefined : performance.now() - respondedAt;
      resolve({ outcome: `${status ?? "no response"}, then ${ending}`, after });
      socket.destroy();
    };
    socket.on("connect", () => {
      socket.write(`POST / HTTP/1.1\r\nHost: localhost\r\nContent-Length: ${bytes}\r\n\r\n${"x".repeat(bytes)}`);
      if (readAfterMs > 0) {
        socket.pause();
        setTimeout(() => socket.resume(), readAfterMs);
      }
    });
    socket.on("data", (chunk) => {
      respondedAt ??= performance.now();
      chunks.push(chunk);
    });
    socket.on("end", () => finish("FIN"));
    socket.on("error", (error) => finish(error.code));
    socket.setTimeout(2000, () => finish("connection still open 2 s later"));
  });
}

async function listening(port) {
  for (let attempt = 0; attempt < 100; attempt++) {
    const open = await new Promise((resolve) => {
      const socket = net.connect(port, "127.0.0.1", () => {
        socket.destroy();
        resolve(true);
      });
      socket.on("error", () => resolve(false));
    });
    if (open) return;
    await new Promise((resolve) => setTimeout(resolve, 100));
  }
  throw new Error(`nothing is listening on ${port}`);
}

const folder = mkdtempSync(path.join(os.tmpdir(), "workerd-repro-"));
writeFileSync(path.join(folder, "config.capnp"), config);
const child = spawn(workerd.default, ["serve", path.join(folder, "config.capnp")], {
  stdio: ["ignore", "inherit", "inherit"],
  windowsHide: true,
});
const reference = http.createServer((request, response) => response.writeHead(413).end("too large"));
await new Promise((resolve) => reference.listen(PORT.node, "127.0.0.1", resolve));
const miniflare = (script) =>
  new Miniflare(convertV4MiniflareOptions({ modules: true, compatibilityDate: "2026-09-01", script }));
const miniflares = { refuse: miniflare(REFUSE), read: miniflare(READ) };

try {
  await Promise.all(Object.values(PORT).map(listening));
  const miniflarePort = async (mf) => Number((await mf.ready).port);
  console.log(`workerd ${workerd.version}, Node ${process.version}, ${os.type()} ${os.release()}; 20 requests per row`);
  const rows = [
    ["workerd: Worker does not read the body", PORT.refuse, 5000],
    ["workerd: Worker forwards through a service binding to that Worker", PORT.forwardToRefuse, 5000],
    ["workerd, control: Worker reads the body", PORT.read, 5000],
    ["workerd, control: Worker forwards through a service binding to the reading Worker", PORT.forwardToRead, 5000],
    ["Miniflare: Worker does not read the body", await miniflarePort(miniflares.refuse), 5000],
    ["Miniflare, control: Worker reads the body", await miniflarePort(miniflares.read), 5000],
    ["Node http server, reference: does not read the body", PORT.node, 5000],
    ["workerd: Worker does not read the body", PORT.refuse, 100000],
  ];
  for (const [name, port, bytes] of rows) {
    for (const readAfterMs of [0, 300]) {
      const outcomes = new Map();
      const after = [];
      for (let i = 0; i < 20; i++) {
        const result = await probe(port, bytes, readAfterMs);
        outcomes.set(result.outcome, (outcomes.get(result.outcome) ?? 0) + 1);
        if (readAfterMs === 0 && result.after !== undefined && !result.outcome.includes("still open")) {
          after.push(result.after);
        }
      }
      const counts = [...outcomes].map(([outcome, n]) => `${n} x ${outcome}`).join("; ");
      const timing = after.length === 0
        ? ""
        : ` (closed ${Math.min(...after).toFixed(1)}-${Math.max(...after).toFixed(1)} ms after the response arrived)`;
      console.log(`${name}, ${bytes}-byte body, client reads after ${readAfterMs} ms: ${counts}${timing}`);
    }
  }
} finally {
  reference.close();
  await Promise.all(Object.values(miniflares).map((mf) => mf.dispose()));
  child.kill();
  await new Promise((resolve) => (child.exitCode !== null ? resolve() : child.once("exit", resolve)));
  rmSync(folder, { recursive: true, force: true });
}
Its output with workerd 1.20260921.1, and with 1.20260923.1
workerd 1.20260921.1, Node v24.14.1, Windows_NT 10.0.26200; 20 requests per row
workerd: Worker does not read the body, 5000-byte body, client reads after 0 ms: 20 x 413, then connection still open 2 s later
workerd: Worker does not read the body, 5000-byte body, client reads after 300 ms: 20 x 413, then connection still open 2 s later
workerd: Worker forwards through a service binding to that Worker, 5000-byte body, client reads after 0 ms: 20 x 413, then ECONNRESET (closed 3.3-4.2 ms after the response arrived)
workerd: Worker forwards through a service binding to that Worker, 5000-byte body, client reads after 300 ms: 20 x no response, then ECONNRESET
workerd, control: Worker reads the body, 5000-byte body, client reads after 0 ms: 20 x 413, then connection still open 2 s later
workerd, control: Worker reads the body, 5000-byte body, client reads after 300 ms: 20 x 413, then connection still open 2 s later
workerd, control: Worker forwards through a service binding to the reading Worker, 5000-byte body, client reads after 0 ms: 20 x 413, then connection still open 2 s later
workerd, control: Worker forwards through a service binding to the reading Worker, 5000-byte body, client reads after 300 ms: 20 x 413, then connection still open 2 s later
Miniflare: Worker does not read the body, 5000-byte body, client reads after 0 ms: 20 x 413, then ECONNRESET (closed 3.2-33.9 ms after the response arrived)
Miniflare: Worker does not read the body, 5000-byte body, client reads after 300 ms: 20 x no response, then ECONNRESET
Miniflare, control: Worker reads the body, 5000-byte body, client reads after 0 ms: 20 x 413, then connection still open 2 s later
Miniflare, control: Worker reads the body, 5000-byte body, client reads after 300 ms: 20 x 413, then connection still open 2 s later
Node http server, reference: does not read the body, 5000-byte body, client reads after 0 ms: 20 x 413, then connection still open 2 s later
Node http server, reference: does not read the body, 5000-byte body, client reads after 300 ms: 20 x 413, then connection still open 2 s later
workerd: Worker does not read the body, 100000-byte body, client reads after 0 ms: 20 x 413, then ECONNRESET (closed 0.1-0.2 ms after the response arrived)
workerd: Worker does not read the body, 100000-byte body, client reads after 300 ms: 20 x no response, then ECONNRESET
workerd 1.20260923.1, Node v24.14.1, Windows_NT 10.0.26200; 20 requests per row
workerd: Worker does not read the body, 5000-byte body, client reads after 0 ms: 20 x 413, then connection still open 2 s later
workerd: Worker does not read the body, 5000-byte body, client reads after 300 ms: 20 x 413, then connection still open 2 s later
workerd: Worker forwards through a service binding to that Worker, 5000-byte body, client reads after 0 ms: 20 x 413, then ECONNRESET (closed 5.1-6.5 ms after the response arrived)
workerd: Worker forwards through a service binding to that Worker, 5000-byte body, client reads after 300 ms: 20 x no response, then ECONNRESET
workerd, control: Worker reads the body, 5000-byte body, client reads after 0 ms: 20 x 413, then connection still open 2 s later
workerd, control: Worker reads the body, 5000-byte body, client reads after 300 ms: 20 x 413, then connection still open 2 s later
workerd, control: Worker forwards through a service binding to the reading Worker, 5000-byte body, client reads after 0 ms: 20 x 413, then connection still open 2 s later
workerd, control: Worker forwards through a service binding to the reading Worker, 5000-byte body, client reads after 300 ms: 20 x 413, then connection still open 2 s later
Miniflare: Worker does not read the body, 5000-byte body, client reads after 0 ms: 20 x 413, then ECONNRESET (closed 5.5-42.3 ms after the response arrived)
Miniflare: Worker does not read the body, 5000-byte body, client reads after 300 ms: 20 x no response, then ECONNRESET
Miniflare, control: Worker reads the body, 5000-byte body, client reads after 0 ms: 20 x 413, then connection still open 2 s later
Miniflare, control: Worker reads the body, 5000-byte body, client reads after 300 ms: 20 x 413, then connection still open 2 s later
Node http server, reference: does not read the body, 5000-byte body, client reads after 0 ms: 20 x 413, then connection still open 2 s later
Node http server, reference: does not read the body, 5000-byte body, client reads after 300 ms: 20 x 413, then connection still open 2 s later
workerd: Worker does not read the body, 100000-byte body, client reads after 0 ms: 20 x 413, then ECONNRESET (closed 0.1-0.2 ms after the response arrived)
workerd: Worker does not read the body, 100000-byte body, client reads after 300 ms: 20 x no response, then ECONNRESET

Activity

  1. workers-devprod commented on Sep 24, 2026

    @workers-devprod
    Contributor

    Automated triage

    Issue assessment

    • Actionability: ✅ Actionable.
    • Type: Bug.
    • Reproduction: Automatic attempt requested.
    • Labels:
      • package:miniflare (high)
      • upstream:workerd (high)
      • windows (medium)
      • package:vite-plugin (medium)
    Actionability and missing information

    Thorough bug report with versions, repro script, and expected/actual behavior: a 413 sent without reading the request body is followed by ECONNRESET within ms when the body crossed a service binding, as Miniflare's entry worker does for every request; workerd is the suspected root cause.

    Reproduction reason: The report supplies a self-contained script with exact commands that deterministically triggers the defect by delaying the client's read, and specifies the observable to verify: ECONNRESET following a 413 when the body crossed a service binding.

    Reproduction

    • Result: ⚠️ Blocked.
    • Summary: Automated reproduction failed before recording a result.
    Evidence and attempted steps

    Blocker: Automated reproduction failed before recording a result.

    Agent outcome: Failed.

    Failure type: submission_timeout

    Failure: Submission exceeded the configured timeout.

    Details: The operation ran longer than the configured durability timeout.

    Submission ID: sub_ik_f3b02abd7da2120e2b7fe613bc1c0bd7


    This summary was generated by an LLM and may contain errors. Please verify its findings.

    Triage bot commands and options

    Repository collaborators with write or admin access can trigger actions by commenting on this issue:

    • @workers-devprod reproduce: Attempt to reproduce the issue using its description and recent comments.
  2. BigCat0922 commented on Oct 9, 2026

    @BigCat0922

    補充同版本 native 與 Wrangler 代理的本機對照(Windows,未知長度 1.1MB 串流)

    這是本機重現,未測 deployed Workers,不宣稱正式平台故障。fixture不含框架、登入、DB、R2、真實專案或憑證。

    固定條件:Node24.19.0、Wrangler4.92.0、Miniflare4.20260515.0、workerd1.20260515.1,compatibility_date=2026-05-15、nodejs_compat。使用相同helper與程式,11×100,000 bytes、無Content-Length;客戶端原timeout5000ms、無重試,各次新persist。

    A:實際Wrangler/Miniflare代理;A0:Miniflare core入口;B:同版本workerd serve原生socket直接映射fixture;C:原生socket經一個本機service binding。另測agent:false的新TCP tuple、獨立Node程序,以及同時正常請求;只證明外部TCP獨立,不宣稱所有內部連線獨立。

    觀察一:B的Worker回413後,客戶端仍可能ECONNRESET而失去回應;其他新/同時正常請求仍可處理。取消時實際剩餘未讀bytes/native grace分支尚未量測,沒有把此現象全部歸因service binding。

    觀察二:A下一GET在outer core/ProxyWorker enter及forward之後,遇Network connection lost,尚未進inner core/fixture Worker即retry-queued。ProxyWorker錯誤分支比較同origin的完整請求URL(含/api/projects/fixture/schedule)與base target(只有/);catch加入GET retry queue,下一個不同ID的新請求才觸發processQueue。

    獨立GET入場5016ms;只縮短下一GET期限為1000ms(POST/agent仍5000ms),入場1016ms。日誌顯示測試端逾時後發出的新請求才喚醒舊GET,不是Worker固定等待5秒。顯式釋放舊socket亦不能保證解決。Wrangler內建重試保留並明列,不把第二次forward算成客戶端成功;原逾時仍FAIL。

    附件保留client request finish/response end/socket connect-close、proxy forward/error/queue、core及Worker enter-return、原helper與有期限候選、固定lock。兩版helper均保留失敗。安裝與執行:npm ci;npm run repro。只綁127.0.0.1,4320–4322被占用即停止;產物為診斷結果,exit0不代表驗收PASS。

    請協助分別確認native取消/回應遺失,以及Wrangler錯誤分類/佇列喚醒是否需由workers-sdk另案處理。我們尚未追到native C++/封包層,也沒有以更換runtime加改cancel的複合對照宣稱修復。

    schedule-body-repro.zip

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions