Skip to content

fix(sessiondict): defer kick out of the registry compute block and guard remove by live register — fixes #304 - #311

Open
ImDanXie wants to merge 2 commits into
apache:mainfrom
ImDanXie:fix/sessiondict-kick-defer
Open

ImDanXie wants to merge 2 commits into
apache:mainfrom
ImDanXie:fix/sessiondict-kick-defer

Conversation

@ImDanXie

Copy link
Copy Markdown
Contributor

Summary

SessionRegistry.add() invokes the kicked register's callback inside the tenantSessions.compute() block. The real register answers kick() by unregistering itself synchronously on the same thread, re-entering compute() for the same key — on the same-owner takeover branch that nested remove() deletes the mapping add() has just installed, silently losing the new session from the dictionary (a subsequent kill cannot find it).

Related failure mode: when a taken-over register is torn down late (its stream is closed by the client only after it receives the Quit), its remove() can delete the mapping of the replacement session registered by the same clientId — session-dictionary counts drift.

Trigger: same-clientId rapid reconnect, routine for diskless clients that reboot (and thus reconnect) frequently.

Fix

Commit 1 (fix):

  • collect the kick and perform it after leaving the compute block (the notification is best-effort; the previous stream may already be closed);
  • remove() only drops the dictionary entry when the removed register is still the live one for that owner, so a delayed teardown cannot delete the new session (or its counters).

Test evidence

Commit 2 (test) adds three regression tests:

  • testKickCallbackReenteringRemoveKeepsNewSession — kick callback re-enters remove() on the same thread; the new session must survive.
  • testReAddSameOwnerWithReenteringKickKeepsNewSession — same owner re-registered with a different register; the nested remove must not delete the fresh mapping.
  • testDelayedTeardownOfTakenOverRegisterKeepsNewSession — the old register's teardown lands after the takeover; the entry and counters must stay with the live register.

All three fail on the unfixed code; the full session-dict suite (18/18) passes with the fix.

Production validation

Deployed in a rolling 6-node cluster (wholesale lib/ replacement, one node at a time, zero rollbacks). The rolling restarts forced all 13 live clients (including the bridge replicas) through same-clientId rapid reconnects; reconnects landed cleanly with no session-dictionary inconsistency observed.

Fixes #304

…ard remove by live register

SessionRegistry.add() invoked the kicked register's callback inside the
tenantSessions.compute() block. The real register answers kick() by
unregistering itself synchronously on the same thread, re-entering compute for
the same key; on the same-owner takeover branch the nested remove() deleted
the mapping the add had just installed, silently losing the new session from
the dictionary (subsequent kill could not find it).

Two changes:
- collect the kick and perform it after leaving the compute block; the
  notification is best-effort (the previous stream may already be closed)
- remove() only drops the dict entry when the removed register is still the
  live one for that owner, so the delayed teardown of a taken-over
  register keeps the new session (and the counters) intact

Fixes apache#304
…eardown

testKickCallbackReenteringRemoveKeepsNewSession and
testReAddSameOwnerWithReenteringKickKeepsNewSession cover a kick callback that
re-enters remove() on the same thread; testDelayedTeardownOfTakenOverRegisterKeepsNewSession
covers the old register's delayed teardown landing after the new register was
added. The first three fail on the unfixed code.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] SessionRegistry kick re-enters the registry compute block and a delayed removal can drop a re-registered session

1 participant