Skip to content

telemetry: log structured direct tool-call timing#30334

Merged
bolinfest merged 1 commit into
mainfrom
pr30334
Jul 2, 2026
Merged

telemetry: log structured direct tool-call timing#30334
bolinfest merged 1 commit into
mainfrom
pr30334

Conversation

@bolinfest

@bolinfest bolinfest commented Jun 27, 2026

Copy link
Copy Markdown
Collaborator

Why

App-server deployments can consume structured JSON logs for operational measurements without requiring an OTEL exporter. Existing tool-result telemetry reports the handler outcome, but it does not separate time spent waiting to dispatch from time spent executing the handler.

A compact completion event for the outer, direct tool call lets consumers measure those phases and correlate them with a conversation and turn. Code-mode calls are intentionally excluded so nested runtime calls do not create overlapping events that are easy to double-count.

What changed

  • Added a ToolCallTimingGuard around direct tool calls. Event-only strings and timing state are captured only when the codex_core::tools::parallel INFO target is enabled.
  • Added a codex.tool_call completion event with conversation, turn, tool, call, trace, dispatch, handler, and total timing fields.
  • Recorded the execution-start marker after the dispatch lock is acquired. Event emission snapshots that marker once so a concurrently starting dispatch cannot produce internally inconsistent fields.
  • Limited the event to ToolCallSource::Direct; unit coverage verifies code-mode calls are ignored.
  • Added cancellation coverage that holds the execution gate and verifies a call cancelled before admission emits exactly one dispatch-only timing event.
  • Added reusable JsonLogCapture support, a JSON-logging-specific TestAppServer constructor, and an end-to-end app-server test that drives a direct exec_command through the public v2 JSON-RPC API and validates the emitted JSON event.

Exec-server-specific request and process timing remains in the stacked PR #30901.

Suggested logging filter

LOG_FORMAT=json \
RUST_LOG='warn,codex_core::tools::parallel=info' \
codex app-server

Event example

Identifier and timing values are illustrative.

codex.tool_call

{
  "timestamp": "2026-06-27T03:45:20.443Z",
  "level": "INFO",
  "fields": {
    "message": "tool call completed",
    "event.name": "codex.tool_call",
    "trace_id": "4bf92f3577b34da6a3ce929d0e0e4736",
    "conversation.id": "67e55044-10b1-426f-9247-bb680e5fe0c8",
    "turn_id": "019f04f8-6ac2-78f1-8625-f04a6d35af18",
    "tool_name": "exec_command",
    "call_id": "call_7b8483",
    "tool_source": "direct",
    "execution_started": true,
    "dispatch_duration_ms": 12,
    "handler_duration_ms": 431,
    "total_duration_ms": 443
  },
  "target": "codex_core::tools::parallel"
}

If execution never starts, execution_started is false, handler_duration_ms is 0, and dispatch_duration_ms covers the full observed lifetime.

If a duration cannot be represented as an unsigned 64-bit millisecond value, all three duration fields are omitted rather than populated with a sentinel that could corrupt downstream calculations.

Test plan

  • just test -p codex-core tool_call_timing_guard_ignores_code_mode_source
  • just test -p codex-core cancellation_before_dispatch_admission_logs_dispatch_only_timing
  • just test -p codex-app-server app_server_emits_structured_tool_call_timing_event

Stack created with Sapling. Best reviewed with ReviewStack.

@bolinfest bolinfest requested a review from a team as a code owner June 27, 2026 03:39
@bolinfest bolinfest force-pushed the pr30334 branch 2 times, most recently from 5e37371 to e9a26a3 Compare June 28, 2026 00:58
@bolinfest bolinfest changed the title telemetry: log structured tool timing events telemetry: log structured tool and inference timing events Jun 28, 2026
@bolinfest bolinfest force-pushed the pr30334 branch 2 times, most recently from f530914 to 39af213 Compare July 2, 2026 05:41
@bolinfest bolinfest changed the title telemetry: log structured tool and inference timing events telemetry: log structured direct tool-call timing Jul 2, 2026
@bolinfest bolinfest changed the base branch from main to pr30942 July 2, 2026 17:55
@bolinfest bolinfest changed the base branch from pr30942 to main July 2, 2026 17:55
@bolinfest bolinfest force-pushed the pr30334 branch 3 times, most recently from e3f0531 to 04de4bb Compare July 2, 2026 18:16
@bolinfest bolinfest force-pushed the pr30334 branch 3 times, most recently from 26ac654 to 38d7926 Compare July 2, 2026 20:12
@bolinfest bolinfest merged commit beca198 into main Jul 2, 2026
50 of 69 checks passed
@bolinfest bolinfest deleted the pr30334 branch July 2, 2026 20:53
@github-actions github-actions Bot locked and limited conversation to collaborators Jul 2, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants