Skip to content

bug(telemetry): crash recorder logs designed refusals and tool exits as crashes #158

Open
Open
bug(telemetry): crash recorder logs designed refusals and tool exits as crashes#158
Labels
bugSomething isn't working

Description

@devswha
devswha
opened on Sep 16, 2026

The crash recorder writes a fingerprinted crash record for ordinary, expected outcomes — user tool refusals, delegation allowlist denials, workflow phase boundaries, non-zero shell exits, rejected edit anchors. Real crashes are buried and the fingerprint index stops deduplicating anything.

Evidence

~/.gjc/agent/gjc-error-events.jsonl on 2026-09-17:

events: 1,992          unique fingerprints: 1,259   (63% unique)
kinds:  {'occurrence': 1992}
names:  Error 1823, ToolError 165, AbortError 4

~/.gjc/agent/gjc-error.log is 341 KB; gjc-error-events.jsonl is 622 KB.

Top recorded "crash" messages:

154   Tool call rejected by user (bash)
149   Tool is outside the parent delegation allowlist.
140   Extension <inline-0> failed: Ralplan planning phase boundary: keep refining the conse…
137   Ultragoal try-harder nudge (1/10) for G001: asking the user was refused before the no…
 26   Edit rejected: N anchors do not match the current file (marked *). The edit was NOT a…
 15   Working directory does not exist: <path>
 12   artifact:<path> URLs cannot be expanded in bash commands…

Bucketed by cause:

other (mostly designed refusals, above)   1,709
shell/command exit                          192
test assertion failure                       50
network/timeout                              31
sqlite query error                            7
cargo/compile failure                         3

Two concrete misclassifications

A designed refusal is not a crash. Tool call rejected by user, Tool is outside the parent delegation allowlist, and the ralplan/ultragoal phase-boundary messages are the system working correctly. That is 580 of 1,992 events.

A tool's non-zero exit is not a crash. A cargo test compile failure was recorded with a full SDK stack trace:

error: could not compile `gajae-core` (lib test) due to 3 previous errors
  --> src/git.rs:1233:21
      None => unsafe { std::env::remove_var("HOME") },
Command exited with code 101
    at wrappedExecute (…/@gajae-code/coding-agent/src/tools/output-meta.ts:1134:13)
    at async execute (…/src/extensibility/extensions/wrapper.ts:142:29)
    at async <anonymous> (…/src/session/agent-session.ts:10258:26)
    …
gjc-crash-record.v1 fp:fb459a7162be028fb05d2e7690694317

The session fixed that compile error 27 seconds later and moved on. The record remains, indistinguishable from a real fault, and its stack trace points at SDK internals rather than at git.rs.

Same for a read-only SQL typo from an interactive session:

[Tool read] Error: no such column: id
gjc-crash-record.v1 fp:38cbb9a4f7b08abc427959d1e5334b0e

Why this is a problem

  • 63% of events carry a unique fingerprint. A deduplicating index that dedupes almost nothing is just an append log with extra steps.
  • gjc-crash-index.json surfaces "recent 21 event ids". With this signal-to-noise, those 21 are recent noise, not recent crashes.
  • The stack traces mislead. I misread the cargo exit above as an ego/SDK crash on first pass precisely because the trace is all SDK frames.

Expected

Do not create a crash record for:

  • user-declined tool calls,
  • policy/allowlist denials and workflow phase boundaries,
  • non-zero exit from a tool that reports command failure through its own result channel,
  • rejected edits whose anchors did not match.

Keep the crash path for unhandled exceptions and invariant violations. Fingerprint on the failing frame, not on the message text, so recurrence actually collapses.

Activity

  1. devswha
    added
    bugSomething isn't working
    on Sep 16, 2026

  2. devswha commented on Sep 16, 2026

    @devswha
    devswha
    on Sep 16, 2026
    OwnerAuthor

    Scope correction after checking where the recorder actually lives: this is not fixable in this repository.

    The crash recorder is SDK-side:

    $ grep -rl "gjc-crash-record\|crash-index" node_modules/@gajae-code/coding-agent/src
node_modules/@gajae-code/coding-agent/src/crash/upstream/relay.ts

$ grep -rn "gjc-crash-record\|crash-index\|gjc-error-events" server/ shared/ src/ scripts/
(no matches)

    Classification, fingerprinting and the ~/.gjc/agent/gjc-error-events.jsonl write all happen inside @gajae-code/coding-agent. The app can set crashReport.upstream / crashReport.upstreamDsn / gjc.crash, but none of those change what counts as a crash — they only control upstream relay.

    The measurements stand and are worth carrying upstream as-is:

    • 1,992 events, 1,259 unique fingerprints (63% unique — the index dedupes almost nothing)
    • 580 of them are designed refusals: user tool declines (154), delegation allowlist denials (149), ralplan phase boundaries (140), ultragoal nudges (137)
    • a cargo test exit 101 recorded with a pure-SDK stack trace, fixed 27s later by the same session

    Keeping this open as an upstream tracking issue rather than an app bug. No app PR will follow.

  3. devswha commented on Sep 19, 2026

    @devswha
    devswha
    on Sep 19, 2026
    OwnerAuthor

    Verified 2026-09-19 against the installed (lifecycle-patched) @gajae-code/coding-agent 0.16.4 and the latest published 0.17.2 (npm-pack compared). Real, and unchanged in 0.17.2.

    Mechanism confirmed in source

    • agent-core/src/telemetry.ts:1883 — every tool span that ends status === "error" calls recordHandledError(\Tool ${toolName}`, errorObject)`. No class of tool error is excluded.
    • session/agent-session.ts (0.16.4 :10242 / 0.17.2 :10367) — a user rejection is thrown as ToolError("Tool call rejected by user (…)"), so the designed refusal is an error-status span.
    • tools/output-meta.ts wrappedExecute rethrows tool failures (incl. non-zero command exits) as new Error(renderError(e)) with an SDK-internal stack, so the record's trace points at output-meta.ts/wrapper.ts, never at the user's command.
    • @gajae-code/utils/postmortem.ts recordHandledError records any Error with a stack and fingerprints name+message+stack together — hence near-unique fingerprints.

    Live repro (installed 0.16.4, agent dir redirected to a temp dir)

    Feeding the recorder the exact error shapes the session throws produced:

    gjc-crash-event.v1 {"n":"ToolError","m":"Tool call rejected by user (bash)","fp":"0500aca5…"}
gjc-crash-event.v1 {"n":"ToolError","m":"Command exited with code 101 error: could not compile `gajae-core` (lib test)","fp":"080b84ec…"}
gjc-crash-event.v1 {"n":"ToolError","m":"Tool call rejected by user (read)","fp":"9973e5bb…"}

    3 events, 3 distinct fingerprints — two of them designed refusals differing only by tool name. gjc-error.log got the matching gjc-crash-record.v1 records with all-SDK stacks. As a bonus, an accidental unhandled SyntaxError in the repro script landed in the fatal gjc-crash.log: fatal and designed-refusal evidence share one store with nothing distinguishing them.

    0.17.2 comparison

    telemetry.ts call site, postmortem.ts recordHandledError (diff clean), and the rejection ToolError throw are all byte-identical in 0.17.2. Nothing in the 0.16.4 → 0.17.2 delta filters refusals, allowlist denials or tool exits out of the crash store.

    Ready to carry upstream as-is; the Expected list in the issue body is still accurate against 0.17.2.

  4. devswha commented on Sep 19, 2026

    @devswha
    devswha
    on Sep 19, 2026
    OwnerAuthor

    Filed upstream after verification against dev @ 2026-09-19 (all cited code paths identical there): Yeachan-Heo/gajae-code#5718

  5. devswha
    added a commit that references this issue on Sep 19, 2026

    docs(release): record beta.18/19 and the post-beta.14 handoff checkpoint

    857f3a8

  6. devswha commented on Sep 25, 2026

    @devswha
    devswha
    on Sep 25, 2026
    OwnerAuthor

    Rechecked on the published 0.17.6 (CLI binary and app SDK) after #176, against a running build.

    • Fixed upstream (#5743): a permission denial and a bash command that exits non-zero are no longer recorded. Checked live through the app: one allowed ls /definitely/missing/path and one denied call, zero new occurrences.
    • Still recorded: hook refusals ({ block: true } becomes a plain Error in the extension wrapper, so our delegation allowlist refusal appears 8× in six hours), EditMatchError, read of an unknown embedded: path and Shell runtime exited with code 70. 12 records, none a real fault.

    Filed upstream as Yeachan-Heo/gajae-code#5938. Separately, 8 of those 12 came from this repo's own Bun test suite writing into the operator's real ~/.gjc journal; the test runner now gives every Bun test file a throwaway HOME (PR to follow). Keeping this open until the upstream follow-up lands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions