Skip to content

Optimize tracing memory usage - #7796

Open
bobrik wants to merge 4 commits into
cloudflare:mainfrom
bobrik:ivan/optimize-tracing
Open

bobrik wants to merge 4 commits into
cloudflare:mainfrom
bobrik:ivan/optimize-tracing

Conversation

@bobrik

@bobrik bobrik commented Oct 17, 2026

Copy link
Copy Markdown
Member

There are noticeable amounts of memory allocated in these two:

  • workerd::IoContext::makeAsyncTraceScope
  • workerd::IoContext::makeUserAsyncTraceScope

Changes here aim to reduce that. In order of appearance:


Fix allocation sampling in benchmarks

Disable sampling with interval zero and verify that the thread-local
sampler adopts interval one before collecting allocation statistics.
This prevents benchmarks from reporting zero allocations.

Add a regression test covering repeated sampling activation.
Validated with five test runs.


Share one async context frame for request tracing

Install internal and user tracing state together at request entry
points, avoiding a second frame allocation and copy of inherited
storage. Preserve separate tracing keys and existing payload lifetimes.

Add regression tests for inheritance, restoration, and captured frames,
plus a benchmark comparing separate and combined scopes.

  • CPU time: 4.83 to 2.38 us/op.
  • Native allocation volume: 896 to 448 B/op.
  • Native allocations: 6 to 3 per operation.
  • Allocation volume also halves with 4 and 16 inherited entries.

Reduce allocations for unobserved tracing scopes

Capture the originating IoContext ID for unobserved internal and user
tracing scopes instead of allocating span payloads and ownership nodes.
Preserve context masking, foreign-request fallback, and cached span
identity. Create a no-op user span only when requested.

Add regression tests for ownership, span identity, and overlapping
requests, plus a benchmark comparing legacy and compact scope state.

  • CPU time: 6.37 to 6.41 us/op, within run-to-run variation.
  • Native allocation volume: 1.12 to 0.92 KiB/op (18.0% less).
  • Native allocations: 8.03 to 5.05 per operation (about 3 fewer).

Store small async context frames inline

Keep up to two storage entries in the frame object and use the existing
hash table for larger contexts. Preserve inheritance, duplicate-key
updates, and dead-key cleanup.

Update memory tracking and add tests for promotion and GC lifetimes.
Extend allocation and lookup benchmarks around the inline threshold.

  • Native allocation volume: 448 to 256 B/op (42.9% less).
  • Native allocations: 3 to 1 per operation.
  • Two-entry lookup: 4.28 to 2.99 ns/op.

Disable sampling with interval zero and verify that the thread-local
sampler adopts interval one before collecting allocation statistics.
This prevents benchmarks from reporting zero allocations.

Add a regression test covering repeated sampling activation.
Validated with five test runs.
Install internal and user tracing state together at request entry
points, avoiding a second frame allocation and copy of inherited
storage. Preserve separate tracing keys and existing payload lifetimes.

Add regression tests for inheritance, restoration, and captured frames,
plus a benchmark comparing separate and combined scopes.

* CPU time: 4.83 to 2.38 us/op.
* Native allocation volume: 896 to 448 B/op.
* Native allocations: 6 to 3 per operation.
* Allocation volume also halves with 4 and 16 inherited entries.
Capture the originating IoContext ID for unobserved internal and user
tracing scopes instead of allocating span payloads and ownership nodes.
Preserve context masking, foreign-request fallback, and cached span
identity. Create a no-op user span only when requested.

Add regression tests for ownership, span identity, and overlapping
requests, plus a benchmark comparing legacy and compact scope state.

* CPU time: 6.37 to 6.41 us/op, within run-to-run variation.
* Native allocation volume: 1.12 to 0.92 KiB/op (18.0% less).
* Native allocations: 8.03 to 5.05 per operation (about 3 fewer).
Keep up to two storage entries in the frame object and use the existing
hash table for larger contexts. Preserve inheritance, duplicate-key
updates, and dead-key cleanup.

Update memory tracking and add tests for promotion and GC lifetimes.
Extend allocation and lookup benchmarks around the inline threshold.

* Native allocation volume: 448 to 256 B/op (42.9% less).
* Native allocations: 3 to 1 per operation.
* Two-entry lookup: 4.28 to 2.99 ns/op.
@bobrik
bobrik requested review from a team as code owners October 17, 2026 00:04
@clanky-the-clanker

clanky-the-clanker Bot commented Oct 17, 2026 •

Copy link
Copy Markdown

✅ Approved — No issues found. Inline frame storage, the combined request trace scope and the compact unobserved scope state keep the existing behaviour.

The decision is advisory: this comment never approves, blocks or requests changes on the pull request.


✅ All checks passed

🔒 Security: ✅ LGTM

No security concerns. Isolation between requests still holds for unobserved scopes: a frame from another request falls back instead of exposing its span.


📖 AGENTS.md: ✅ LGTM

No AGENTS.md updates are needed for this change.


📋 Code Quality: ✅ LGTM

I checked the production changes in IoContext trace scopes, worker-entrypoint, api/tracing.c++ and AsyncContextFrame, and found no defects.

  • AsyncContextFrame inline storage:
    • Inherited entries are copied through upsertStorageEntry.
    • Dead keys are skipped on insert and cleared in both the inline and the table representations.
    • The two inline slots are promoted to the hash table when a third distinct key arrives. A duplicate key updates its value in place.
    • get and the upsert path both match keys by pointer identity, so they agree.
  • Unobserved scopes:
    • These scopes now keep only the ID of the IoContext they came from.
    • When that ID is the current context, getCurrentTraceSpan and getCurrentUserTraceSpan return an unobserved span, as the old dummy span did.
    • Frames captured from a different request fall back the same way they did before.
  • Request entry points: Every place that built the internal and user scopes separately now calls makeRequestTraceScope. No caller of the old two-step setup is left.

⚡ Performance: ✅ LGTM

The changes cut down allocations as described. The linear search over the two inline slots is cheaper than a hash lookup at this size.


📝 Documentation: ✅ LGTM

The docs/jsg.md update matches the new storage behaviour.


Reviewed commit: 882f24d5

@jasnell

jasnell commented Oct 17, 2026

Copy link
Copy Markdown
Collaborator

clanky review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants