[stack 4/8] feat(trace): opt-in trace store linked to usage.jsonl

[OnlineChef (ChefGroep)]

Stacked re-landing of #298 (feat/trace-store). Position 4/8; base: stack/20261005-03-gui-redesign. Merge bottom-up; each layer is a real merge of the original PR head, so the diff shows only that PR plus any conflict resolution (noted in the merge commit message).

Stack

#	Layer	Original
1/8	#304 feat: make prompt cache behavior measurable	#296
2/8	#305 feat: expand providers and make native model discovery account-aware	#297
3/8	#306 feat(gui): start auth-inspired OpenCodex redesign	#299
4/8	#307 feat(trace): opt-in trace store linked to usage.jsonl	#298
5/8	#308 feat(trace): add safe local trace reader	#300
6/8	#309 feat(trace): cover compact and response-cache hits	#301
7/8	#310 feat(trace): cover Responses WebSocket turns	#302
8/8	#311 feat(trace): cover live call-create HTTP	#303

---

Original description:

What

usage.jsonl stays the compact telemetry SOT. Full prompts/responses go to a separate local store, trace.sqlite, linked by traceId (= requestId). Off by default.

Modes (OCX_TRACE)

• off (default): nothing read or stored.
• metadata: no bodies stored; each usage row gets a payload-free trace object: byte sizes, message/tool-call/tool-def/attachment counts, request/outbound/response hashes, plus systemHash, toolsHash, prefixHash of the final provider wire body.
• redacted: also stores inbound request, final provider body and response (gzip BLOBs), secrets stripped with the existing lib/redact.
• full: verbatim bodies, explicit opt-in.

Knobs: OCX_TRACE_TTL_HOURS (24), OCX_TRACE_MAX_BODY_BYTES (512 KiB), OCX_TRACE_MAX_DB_MB (256, oldest evicted first), OCX_TRACE_SAMPLE (1.0).

Why the section hashes

For cache-hit debugging: two consecutive turns with equal prefixHash but a moved systemHash or toolsHash tells you which section broke the provider prompt-cache prefix, without storing any content.

Wiring

• src/trace/{types,settings,store,capture}.ts (new). Store is bun:sqlite, WAL, TTL + size-cap pruning, every entry point swallows its own failures.
• /v1/responses, /v1/messages, /v1/chat/completions: beginTrace (reads a clone of the inbound body) + runWithTrace (AsyncLocalStorage).
• fetchWithHeaderTimeout records the final wire body via noteOutboundRequestBody (last body wins on retries; outboundCount records how many were sent).
• inspectResponseLogJson / inspectResponseLogSsePayload feed the bounded response copy.
• addFinalRequestLog finalizes the trace and stamps traceId + trace on the usage row; normalizeUsageEntry whitelists/validates them.
• Docs: reference/cli.md.

Verified locally

• bun test: new tests/trace.test.ts (16) plus usage-log, request-log, usage-failure-persistence, usage-surfaces, usage-log-metrics, debug-settings, response-cache-e2e, api-usage, adapter-usage, chat-completions-endpoint, claude-messages-endpoint, openai-responses-passthrough, client-server-route-gate: all green.
• bun run privacy:scan: passed.
• Not run locally: bun run typecheck and the full suite (the 1 GB sandbox OOMs on tsc). Relying on CI for those; please treat red CI as authoritative.

Known limits (follow-ups)

• Outbound capture only covers adapters that send through fetchWithHeaderTimeout; adapters with their own fetch show outboundBytes absent.
• /v1/responses/compact, WebSocket and live/realtime routes are not wired.
• redacted is pattern-based: free-text secrets in prompts that match no known pattern are not detectable.
• No CLI/management-API reader yet; use readTrace/listTraces or sqlite3 on trace.sqlite.
• Cache-hit/replay paths (logCacheHitRequest) are not traced.

Summary by CodeRabbit

• New Features
  • Added optional request tracing, disabled by default, with metadata-only, redacted, and full-body modes.
  • Traces can capture request and response details, with configurable retention, storage limits, and sampling.
  • Request logs can include trace IDs and payload-free trace summaries.
• Documentation
  • Documented trace modes, configuration, captured traffic, storage limits, and privacy considerations.

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

• Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[OnlineChef (ChefGroep)]

Review note (non-blocking): in redacted mode appendTraceResponse runs redactBodyForStorage on each streamed chunk separately. A secret split across two SSE chunks would not match the redaction patterns in either half, so it could be stored unredacted. Inbound/outbound bodies are redacted whole, so this only affects streamed responses. Suggest buffering and redacting the assembled response before it is persisted (or redacting once in finalize). Everything else in the trace layer looks solid: default off, 0700/0600 permissions, TTL plus size caps, parameterized SQL. Note that stack PRs on non-main bases only get the label/Socket/react-doctor checks; the full CI (tests, typecheck, CodeQL) runs once a PR is retargeted to main.