Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -150,9 +150,12 @@ Alternatives considered and rejected:
operations of that batch unapplied, where separate batches were independent.
Chunking at the wire cap bounds a batch, and the ledger already models
`unprocessed`.
- Human operations settled `undeliverable` or `unconfirmed` (at unbind, at
`doc_reset`, on a dead transport) are still reported only to the dev panel;
surfacing them to the user is a follow-up.
- Human operations settled `undeliverable`, `unconfirmed`, or
`unacknowledged` produce one `reportError` call and a dev-panel entry per

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (non-blocking): the code caps platform reports at 3 per 60 s per sender (mayReportLostEdit). The dev-panel entry is the only per-batch record. doc_reset drains share that cap, so a reset of 3 or more batches can hide a real timeout or transport loss in the same window, and filtering out code:doc_reset won't bring it back. Worth stating the cap here, and either keeping doc_reset out of it or saying it shares it.

batch. Reports include the mint-time workflow id and termination cause.
`doc_reset` identifies batches superseded by the draft under decision 4,
so lost-edit queries can exclude them. User-facing notices for sender
termination remain a follow-up.
- Two browser tabs on the same document, both agent-bound, are out of scope:
which tab's draft wins a divergence is decided by the server's sequence
check, not by anything the frontend promises.
Expand Down
4 changes: 3 additions & 1 deletion src/platform/telemetry/reportError.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -72,12 +72,14 @@ describe('reportError', () => {

reportError(error, {
surface: 'platform',
errorType: 'workspace_auth_gate_initialization_failure'
errorType: 'workspace_auth_gate_initialization_failure',
fingerprint: ['workspace_auth_gate_initialization_failure']
})

expect(captureException).toHaveBeenCalledWith(
error,
expect.objectContaining({
fingerprint: ['workspace_auth_gate_initialization_failure'],
tags: expect.objectContaining({ bucket: 'canary' })
})
)
Expand Down
10 changes: 7 additions & 3 deletions src/platform/telemetry/reportError.ts
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ export interface ReportErrorOptions {
errorType: string
/** Product surface responsible for acting on this failure. */
surface: Surface
fingerprint?: string[]
tags?: Record<string, string | number | boolean | undefined>
context?: Record<string, unknown>
level?: 'warning' | 'error'
Expand Down Expand Up @@ -171,13 +172,15 @@ function dispatchToSentry(
surface: Surface,
tags: Record<string, string | number | boolean>,
context: Record<string, unknown>,
level: ReportErrorOptions['level']
level: ReportErrorOptions['level'],
fingerprint: ReportErrorOptions['fingerprint']
): boolean {
try {
captureException(error, {
tags: { ...tags, error_type: errorType, surface },
extra: context,
level
level,
...(fingerprint ? { fingerprint } : {})
})
return true
} catch (reporterFailure) {
Expand Down Expand Up @@ -249,7 +252,8 @@ function dispatch(
surface,
tags,
context,
level
level,
options.fingerprint
)
}
if (datadogLive) {
Expand Down
4 changes: 4 additions & 0 deletions src/workbench/extensions/agent/crdt/devPanelLog.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -199,20 +199,24 @@ describe('devPanelLog', () => {
it('keeps the structural diagnosis on failed frames', () => {
recordDevEvent('doc_ops_result', {
workflowId: 'wf-1',
graphId: '1ff39644-78b1-48bb-9509-53b671c672f1',
ok: false,
applied: [],
skipped: ['op-1'],
code: 'op_failed',
operations: ['set_widget', 'delete_node', 'private user prose'],
message: 'unknown node 7',
failed: { op_id: 'op-1', code: 'unknown_node', message: 'unknown node 7' }
})

expect(devEvents.value[0]?.detail).toEqual({
workflowId: 'wf-1',
graphId: '1ff39644-78b1-48bb-9509-53b671c672f1',
ok: false,
applied: [],
skipped: ['op-1'],
code: 'op_failed',
operations: ['set_widget', 'delete_node', 'String(18)'],
message: 'String(14)',
failed: { op_id: 'op-1', code: 'unknown_node', message: 'String(14)' }
})
Expand Down
4 changes: 4 additions & 0 deletions src/workbench/extensions/agent/crdt/devPanelLog.ts
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,8 @@ export const DEV_EVENT_KINDS = [
'doc_ops_result',
'doc_ops_result_dropped',
'human_ops_settled',
'mint_divergence',
'host_rejected',
'doc_reset',
'doc_nodes_changed',
'schema_error',
Expand Down Expand Up @@ -96,6 +98,7 @@ const STRUCTURAL_STRING_KEYS = new Set([
'doc_id',
'failed',
'frame_type',
'graph_id',
'id',
'kind',
'level',
Expand All @@ -104,6 +107,7 @@ const STRUCTURAL_STRING_KEYS = new Set([
'op',
'op_id',
'op_ids',
'operations',

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: the mint_divergence context now carries graphId. That key normalizes to graph_id, which isn't in this list, so the dev panel summarizes it as String(36) (a scratch call through reportLostEdit confirmed the summarizing). Adding graph_id here would keep the graph of the unresolved owner visible.


Generated by Claude Code

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in e1a7b74ed0: added graph_id to the structural string keys. The existing sanitizer test now includes a UUID under camel-case graphId; it received String(36) before the fix and retains the ID afterward. Structural-value and credential filters still apply.

'outcome',
'phase',
'prompt_id',
Expand Down
55 changes: 43 additions & 12 deletions src/workbench/extensions/agent/crdt/docOpMinter.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -258,6 +258,31 @@ describe('attachDocOpMinter', () => {
}
})

it('reports mint divergence once when a node cannot be serialized', async () => {
const node = new TestSource()
graph.add(node)
vi.spyOn(node, 'serialize').mockImplementation(() => {
throw new Error('private widget content')
})

await afterFlush()
await afterFlush()

expect(minted).toEqual([])
expect(reportError).toHaveBeenCalledExactlyOnceWith(
new Error('agent_crdt_node_snapshot_missing'),
expect.objectContaining({
fingerprint: ['agent_crdt_node_snapshot_missing'],
context: expect.objectContaining({
code: 'missing_node_snapshot',
graphId: graph.id,
nodeId: node.id,
operations: ['add_node']
})
})
)
})

describe('mints one op per graph command', () => {
it.for([
{
Expand Down Expand Up @@ -2047,21 +2072,27 @@ describe('attachDocOpMinter', () => {
expect(
vi.mocked(reportError).mock.calls.map(([, options]) => options)
).toEqual([
{
surface: 'agent',
expect.objectContaining({
errorType: 'agent_crdt_widget_owner_unresolved',
context: { graphId: staleGraph.id, nodeId: stale.id }
},
{
surface: 'agent',
context: expect.objectContaining({
graphId: staleGraph.id,
nodeId: stale.id
})
}),
expect.objectContaining({
errorType: 'agent_crdt_widget_owner_unresolved',
context: { graphId: otherGraph.id, nodeId: other.id }
},
{
surface: 'agent',
context: expect.objectContaining({
graphId: otherGraph.id,
nodeId: other.id
})
}),
expect.objectContaining({
errorType: 'agent_crdt_widget_owner_unresolved',
context: { graphId: staleGraph.id, nodeId: sibling.id }
}
context: expect.objectContaining({
graphId: staleGraph.id,
nodeId: sibling.id
})
})
])
})

Expand Down
46 changes: 26 additions & 20 deletions src/workbench/extensions/agent/crdt/docOpMinter.ts
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ import {
} from './agentCrdtDocLifecycle'
import type { GraphOperation } from './graphOperations'
import type { DocPromotedWidgets } from './agentSubgraphDefinitions'
import { reportLostEdit } from './reportLostEdit'

export interface DocOpMinterDeps {
/** Slice 00's product gate. */
Expand Down Expand Up @@ -446,12 +447,15 @@ export function attachDocOpMinter(deps: DocOpMinterDeps): DocOpMinter {
if (node.graph !== graph) return null
const snapshot = wireNodeSnapshot(node)
if (!snapshot) {
reportOnce(
`snapshot:${nodeKey(graph.id, node.id)}`,
'Agent node add has no snapshot',
'agent_crdt_node_snapshot_missing',
{ graphId: graph.id, nodeId: node.id }
)
const key = `snapshot:${nodeKey(graph.id, node.id)}`
if (reported.has(key)) return null

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpick (non-blocking): the has/add/return guard from reportOnce is now copied here and in reportUnresolvedWidgetOwner. A small shared helper, or letting reportOnce take the reporter, would keep one copy.

reported.add(key)
reportLostEdit('mint_divergence', [{ op: 'add_node' }], {
code: 'missing_node_snapshot',
errorType: 'agent_crdt_node_snapshot_missing',
graphId: graph.id,
nodeId: node.id
})
return null
}
return {
Expand Down Expand Up @@ -537,7 +541,7 @@ export function attachDocOpMinter(deps: DocOpMinterDeps): DocOpMinter {
const rootGraphId = deps.boundRootGraphId() ?? graph.id
const routed = routedWidgetOperation(graph, rootGraphId, event, owner, deps)
if ('unresolvedGraphId' in routed) {
reportUnresolvedOwner(routed.unresolvedGraphId, event)
reportUnresolvedWidgetOwner(routed.unresolvedGraphId, event)
return
}
if ('reason' in routed) {
Expand All @@ -547,6 +551,21 @@ export function attachDocOpMinter(deps: DocOpMinterDeps): DocOpMinter {
schedule({ kind: 'op', operation: routed })
}

function reportUnresolvedWidgetOwner(
graphId: string,
event: IntentOf<'set_widget'>
): void {
const key = `owner:${nodeKey(graphId, event.nodeId)}`
if (reportedRefusals.has(key)) return
reportedRefusals.add(key)
reportLostEdit('mint_divergence', [{ op: 'set_widget' }], {
code: 'unresolvable_widget_owner',
errorType: 'agent_crdt_widget_owner_unresolved',
graphId,
nodeId: event.nodeId
})
}

function takeHeldWrite(
event: IntentOf<'set_widget'>
): HeldPromotedWrite | null {
Expand All @@ -556,19 +575,6 @@ export function attachDocOpMinter(deps: DocOpMinterDeps): DocOpMinter {
return held
}

function reportUnresolvedOwner(
graphId: string,
event: IntentOf<'set_widget'>
): void {
reportOnce(
`owner:${nodeKey(graphId, event.nodeId)}`,
'Agent widget write has an unresolvable owner',
'agent_crdt_widget_owner_unresolved',
{ graphId, nodeId: event.nodeId },
reportedRefusals
)
}

function holdOrRefuseWidgetWrite(
rootGraphId: string,
event: IntentOf<'set_widget'>,
Expand Down
Loading
Loading