Skip to content

feat(telemetry): make Datadog the usable exception sink - #1541

Merged
christian-byrne merged 1 commit into
mainfrom
christian-byrne/tele-datadog-exception-detail
Sep 20, 2026
Merged

christian-byrne merged 1 commit into
mainfrom
christian-byrne/tele-datadog-exception-detail

Conversation

@christian-byrne

@christian-byrne christian-byrne commented Sep 20, 2026 •

Copy link
Copy Markdown
Contributor

Exceptions reached Datadog already, but as a fixed string, so no monitor could tell one failure from another. This makes that payload usable and moves the PostHog copy behind an opt-in.

What was wrong

reportRendererError builds a scrubbed, length-capped Error from the forwarded payload, then throws it away:

const datadogError = new Error('Desktop application exception')
datadogError.name = 'DesktopTelemetryError'
datadogError.stack = undefined
datadogRum.addError(datadogError, { origin, source, forwarded_source, level, reason, exitCode, exit_code, type })

No message, no stack, and no error_type. Every distinct failure mode collapsed into one indistinguishable Datadog error. You could alert on "desktop exceptions rose" and never on which one. That matters now that the hosted frontend routes every reportError() through this path (#1485).

Changes

  • forwardExceptionToRenderer takes the scrubbed error and forwards its message and stack. Context keys stay allow-listed; error_type is added to that list. Free-form diagnostics still never reach Datadog.
  • The renderer reports the scrubbed error it already built instead of the fixed string, and passes error_type through.
  • The PostHog exception copy is opt-in behind POSTHOG_EXCEPTIONS=1. Both sinks carried the same scrubbed error and PostHog bills per event.

Deliberate reversal, flagged for review

#1271 intentionally sent no message or stack to Datadog ("Forward scrubbed exception context (reason/exitCode/type) to the renderer Datadog mirror; keys stay allow-listed"). This reverses that half, on purpose, because Datadog is now the primary exception sink rather than a secondary mirror. Sending nothing identifiable was defensible when PostHog held the detail; it is not once PostHog is opt-out.

No new unscrubbed surface: telemetry.ts is the only dd-error producer, the payload is deliverException's safeError (already scrubAll-ed and capped), and reportRendererError scrubs again on receipt. This is the same text PostHog already received.

Change breakdown

Total changed lines 143 (118 added, 25 deleted).

Category Files Paths Added Deleted Changed Share
Product code 4 src/main/lib/telemetry.ts, src/renderer/src/lib/rendererBootstrap.ts, src/shared/posthogConfig.ts, src/types/ipc.ts 64 24 88 61.5%
Test code 1 src/main/lib/telemetry.test.ts 54 1 55 38.5%

No documentation, configuration, generated files, lockfiles, or vendored code. No merge-only changes.

The product-code count is inflated by comment and docstring updates inside those four files: the module header's provider-split section, the DatadogForwardedError.skipPostHog doc, and the forwardExceptionToRenderer docstring all described the old behaviour and would have been actively misleading if left. The executable change is roughly 20 lines.

Verification

  • Full unit suite: 276 files, 5100 passed, 2 skipped.
  • pnpm typecheck (node, web, e2e, integration) clean; pnpm lint clean; prettier --check clean.
  • Both new tests were control-checked: reverting each half of the fix turns the matching test red, restoring it turns it green.

New tests:

  • forwards the scrubbed message, stack and error_type so a monitor can tell failures apart
  • suppresses the PostHog exception copy unless POSTHOG_EXCEPTIONS opts in, while the Datadog forward still happens

The existing "forwards exceptions to Datadog without message, stack, or arbitrary context" test is renamed to "forwards a bare notice to Datadog when no error is supplied" and kept: it still pins the no-error call path and the context allow-list.

Justification: Merged without pre-merge approval under release time pressure, authorized offline by the author, to complete the Desktop error-reporting path landed in #1485 and Comfy-Org/ComfyUI_frontend#16917 before the release window closes. No human has reviewed this PR. Risk is limited: no new unscrubbed data reaches any sink, the only behaviour change to an existing path is richer detail on an error already being sent, full unit suite and all gates green. Post-merge peer review is owed.

The renderer discarded the scrubbed error it had already built and
reported a fixed "Desktop application exception" with no stack and no
error_type, so every distinct failure collapsed into one Datadog error a
monitor could not act on. Carry the scrubbed message, stack and the
frontend's error_type slug through instead; context keys stay
allow-listed, so free-form diagnostics still never reach Datadog.

With Datadog carrying real detail, the PostHog copy is now opt-in behind
POSTHOG_EXCEPTIONS: both sinks hold the same scrubbed error and PostHog
bills per event.
@coderabbitai

coderabbitai Bot commented Sep 20, 2026 •

Copy link
Copy Markdown

Review Change StackReview Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Team

Run ID: 41db1cb1-ddcb-4e3b-abb0-a0b5fcc744b7

📥 Commits

Reviewing files that changed from the base of the PR and between d941d39 and 6f021f9.

📒 Files selected for processing (5)
  • src/main/lib/telemetry.test.ts
  • src/main/lib/telemetry.ts
  • src/renderer/src/lib/rendererBootstrap.ts
  • src/shared/posthogConfig.ts
  • src/types/ipc.ts

Included review availability: Your plan provides up to 10 included reviews per hour; 1 remains after this review.


📝 Walkthrough

Walkthrough

Exception telemetry now preserves scrubbed error details for renderer reporting. Datadog receives exceptions independently of the action allow-list. PostHog exception capture is opt-in through POSTHOG_EXCEPTIONS.

Changes

Exception telemetry

Layer / File(s) Summary
Flag-controlled exception routing and payloads
src/shared/posthogConfig.ts, src/main/lib/telemetry.ts
Adds parsing for enabled PostHog flag values. Main telemetry conditionally sends exceptions to PostHog and forwards scrubbed messages, stacks, and error_type to the renderer.
Renderer reporting contract
src/renderer/src/lib/rendererBootstrap.ts, src/types/ipc.ts
Renderer reporting preserves the scrubbed original error and its stack. IPC documentation clarifies Datadog-only forwarding.
Telemetry tests
src/main/lib/telemetry.test.ts
Tests cover default flag setup, cleanup, preserved diagnostics, error_type, and suppression of PostHog capture while retaining Datadog forwarding.

Sequence Diagram(s)

sequenceDiagram
  participant Renderer
  participant MainTelemetry
  participant Datadog
  participant PostHog
  Renderer->>MainTelemetry: report scrubbed exception
  MainTelemetry->>Datadog: forward exception details
  MainTelemetry->>PostHog: capture exception when POSTHOG_EXCEPTIONS is enabled
Loading

Suggested reviewers: kosinkadink

Priority: ➖ Normal

Merge Risk: ⚪ Minimal · up to 6f021

Datadog exception forwarding remains available as intended, and no actionable merge-blocking risk remains.

🚥 Pre-merge checks | ✅ 2
✅ Passed checks (2 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
✨ Simplify code
  • Commit to this branch
  • Create a new PR

Comment @coderabbitai help to get the list of available commands.

@christian-byrne
christian-byrne merged commit 787b595 into main Sep 20, 2026
15 checks passed
@christian-byrne
christian-byrne deleted the christian-byrne/tele-datadog-exception-detail branch September 20, 2026 03:38
@christian-byrne

Copy link
Copy Markdown
Contributor Author

Admin-merged without waiting for in-band approval, same release window as #1485 and Comfy-Org/ComfyUI_frontend#16917. Authorized offline by me as the author.

Stating it plainly: no human has reviewed this PR. A post-merge peer review is owed under CC8.1. CI was 15/15 green at merge, and the full unit suite (276 files, 5100 tests) passed locally.

The one thing worth a reviewer's eye is the deliberate reversal of #1271's "no message or stack to Datadog" decision, explained in the PR body. Short version: that was right when PostHog held the error detail, and wrong now that Datadog is the primary exception sink and PostHog is opt-out. No new unscrubbed data reaches Datadog, it is the same scrubbed text PostHog was already receiving.

@huang47 @benjcooley if either of you has ten minutes, the diff is ~20 executable lines across telemetry.ts and rendererBootstrap.ts.

@github-actions github-actions Bot locked and limited conversation to collaborators Sep 20, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant