Skip to content

fix(agent): WARN when an agent closes without completing the handshake - #217

Merged
mikhailm-coder merged 1 commit into
masterfrom
hotfix/mesh-handshake-incomplete-warn
Oct 8, 2026
Merged

mikhailm-coder merged 1 commit into
masterfrom
hotfix/mesh-handshake-incomplete-warn

Conversation

@mikhailm-coder

@mikhailm-coder mikhailm-coder commented Oct 7, 2026 •

Copy link
Copy Markdown

Summary

An agent that upgrades, sends frames but never reaches the authenticated state used to leave no trace at the default log level, while the server kept the half-open session by design (fix256, 2026-10-07: the agent's cmd 2 never arrived, node never created, 15 h offline, nothing in the pod log).

On close of such a connection, meshagent.js now logs one WARN line, visible at the tenant default MESH_LOGGING=INFO:

  • stage 0 (nothing verified): the receivedCommands bitmask decoded, e.g. got cmd1 (auth request), cmd3 (agent info), cmd4 (auth confirm, optional); missing cmd2 (cert+signature);
  • stage 1 (cert verified, never completed): cert verified but connection never completed plus the hold reason recorded by setAgentIssue (unknown device group, bad signature, duplicate agent, ...);
  • always: remote address, frame count, authenticated value, lifetime.

Details:

  • handshake progress lives in obj.diagStage and the address is snapshotted at connect, because obj.close() deletes nodeid/remoteaddrport before the 'close' event fires (the close listener survives removeAllListeners([...])), so authenticated agents closed by the server are never miscounted;
  • WARN lines are capped at 20 per minute with a suppressed count carried into the next line;
  • the existing DEBUG close line stays unconditional;
  • new agentStats.agentHandshakeIncompleteCount, exported to Prometheus as HandshakeIncomplete.

Log only. No handshake deadline and no socket close, as agreed in the Slack thread.

Test

Drove the real CreateMeshAgent close handler with stubbed sockets: fix256 shape warns with missing cmd2; a held-after-verification server close warns with issue=invalidDomainMesh2 and an intact address; an authenticated agent closed by the server stays DEBUG-only; a zero-frame upgrade stays silent; the throttle logs 20 of 25 and reports +5 similar suppressed. node --check passes on all three files.

CU-17tkuw5vmhq
Change-Set: mesh-handshake-incomplete-warn

🤖 Generated with Claude Code

Change set flamingo-stack/mesh-handshake-incomplete-warn: this pull request is the only one in it so far. Another pull request joins by naming this one in a Depends-On line, or by carrying the same Change-Set line.

Linked work

  • …and 1 linked item not shown here

Linked by the Depends-On / Change-Set lines in these descriptions; this block is maintained by the hub.

An agent that upgrades, sends frames but never reaches the authenticated state used to
leave no trace at the default log level, while the server kept the half-open session by
design. On close, log one WARN line with the received-command bitmask decoded (which of
cmd 1/2/3/4 arrived, which is missing) or, once the cert is verified, the hold reason
from setAgentIssue; include the remote address, frame count, authenticated value and
lifetime, count it in agentStats and export it to Prometheus as HandshakeIncomplete.

Progress is tracked in obj.diagStage (0 nothing verified, 1 cert verified, 2 fully
authenticated) and the address is snapshotted at connect, because obj.close() deletes
nodeid/remoteaddrport before the 'close' event fires. WARN lines are capped at 20 per
minute with a suppressed count. The existing DEBUG close line stays unconditional.
No behaviour change.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@mikhailm-coder

Copy link
Copy Markdown
Author

@flamingo-review

@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown

🦩 Flamingo Code Review

✅ No findings on the current head.

Advisory: findings do not block the merge.

3 possible problems checked and ruled out
  • meshagent.js:29 Module-level handshake WARN rate-limit state is shared/global across all domains and server instances in-process: The module-scope rate limiter is indeed shared across all domains/instances in one process, a real but minor cross-tenant log-suppression edge case for a diagnostic WARN feature.
  • meshagent.js:619 diagStage never reaches 1 unless the cert-verified branch at line 1250 executes before close, so 'cert verified but connection never completed' message may be unreachable or mis-ordered relative to receivedCommands cleanup: Speculative concern about a hypothetical other deletion path not shown to exist; receivedCommands is only deleted in the diagStage=1 branch shown, and the why logic correctly distinguishes stage 1 from stage 0 as the candidate itself notes.
  • meshagent.js:1250 obj.receivedCommands is deleted before describeHandshake() reads it on the close path for already-verified agents: Candidate itself concludes the diff is internally consistent and the two paths are mutually exclusive; purely speculative concern about hypothetical future code.

Review again. New commits are not reviewed until you ask:

  • Review the new commits: only what was pushed since this review
  • Review the whole diff again: everything, including what was already reviewed

Or comment @flamingo-review (new commits) or @flamingo-review full (everything). Add the flamingo-review-always label to review every push.

Started 2026-10-07 21:25 UTC · updated 2026-10-07 21:26 UTC · workflow run

@mikhailm-coder
mikhailm-coder merged commit c0dd686 into master Oct 8, 2026
16 of 17 checks passed
@mikhailm-coder
mikhailm-coder deleted the hotfix/mesh-handshake-incomplete-warn branch October 8, 2026 10:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants