Skip to content

πŸ› Bug Report β€” Runtime APIs: traced Durable Object RPC from an alarm holds the session open until the caller's next event (durable_object_subrequest force-closed)Β #7609

Open

Description

@DivMode

With Workers Traces enabled, an RPC call from one Durable Object's alarm to another Durable Object keeps the session open until the caller's next event. The caller's durable_object_subrequest span is then force-closed at the end of every alarm, with span_not_ended: "Span was still open when the invocation ended; it has been force-closed". The callee's jsrpc invocation is recorded as lasting until the caller's next alarm, even though the call itself returns in a few milliseconds.

Setup

  • Production, compatibility date 2026-09-08 with nodejs_compat, Workers Traces enabled.
  • Durable Object A runs an alarm every 500 ms. Each alarm makes one RPC call to Durable Object B, on a stub obtained once from env.B.getByName(...) and reused, and awaits it.
  • The method returns a plain object (a list of quotes); it has no stubs and no disposer. A then sets the next alarm. Both classes are in the same Worker script.

Observed (production traces)

Per alarm, on the caller (A):

Span Duration Ended normally
alarm invocation about 8 ms (wall time about 25 ms) yes
jsRpcCall about 4 ms yes
jsRpcSession about 4 ms yes
durable_object_subrequest β€” no: span_not_ended, force-closed at the end of the alarm

On the callee (B):

Span Duration
jsRpcCall 0 ms
jsrpc invocation, outcome ok 502–506 ms in 958 of 1,000 samples (median 504 ms), so it ends when A's next alarm runs

This happens on every tick, about 20 times a minute. A Worker's fetch calls into the same Durable Objects are never affected: 504 of 504 subrequest spans ended normally.

Not reproducible locally

The same code in local workerd (miniflare 5.20260911.0-alpha, untraced) ends the callee's jsrpc invocation in 2–4 ms. That was measured as the time from the event's eventTimestamp to delivery of its tail event. Disposing the call's promise or its result made no difference locally.

From reading worker-rpc.c++ at 925464ba9f: for a result with no callee-side disposer, deserializeRpcReturnValue drops the call pipeline, so disposing the result releases nothing. JsRpcPromise::dispose only clears originatingCall, which holds span observers. The caller's call and session spans close at about 4 ms while the callee's session outcome comes about 500 ms later. That suggests a capability crossing the session's CompletionMembrane is released only on the caller's next event, perhaps only when caller and callee are on different machines.

Minimal shape

import { DurableObject } from "cloudflare:workers";

export class Callee extends DurableObject {
  quotes() {
    return { status: "live", quotes: [1, 2, 3] };
  }
}

export class Caller extends DurableObject {
  async alarm() {
    this.stub ??= this.env.CALLEE.getByName("callee");
    await this.stub.quotes();
    await this.ctx.storage.setAlarm(Date.now() + 500);
  }
}

Expected

The callee's jsrpc invocation ends when the call returns, and the caller's durable_object_subrequest span ends before the caller's alarm does.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions