Skip to content

Wait for snapshot publication on area/zone status reads - #144

Merged
mitchmitchell merged 52 commits into
mainfrom
cursor/wait-snapshot-publication-area-zone-8a20
Nov 16, 2026
Merged

mitchmitchell merged 52 commits into
mainfrom
cursor/wait-snapshot-publication-area-zone-8a20

Conversation

@mitchmitchell

@mitchmitchell mitchmitchell commented Nov 15, 2026 •

Copy link
Copy Markdown
Owner

Fixes for verification blockers (tip 8bcd55d)

Blocker 1 — publication future drain

_consume_done_publication_future no longer uses suppress(BaseException). It handles cancelled() explicitly and retrieves failures via exception(). Consumption also runs immediately after sent_event so a done publication is drained even when the transport reply future is already complete.

Blocker 2 — [close_reconnect_area_get_status]

Dispatch-only reply (area_id: 0), hold_dispatch_publication, and close_before_execute without pre-calling _ensure_kernel_subscription in the harness. Production now calls _ensure_kernel_subscription() at the start of _async_execute_impl after reconnect.

Blocker 3 — fail-on-old @ 8befca3

  • drop_dispatch_correlation spy only when assert_dispatch_clear_on_success.
  • Session-loss cases drive _on_session_disconnected while deferred snapshot publication is still pending; connection-state events are not deferred.
  • Fail-on-old worktree uses PYTHONPATH=src on old source.

Blocker 4 — mutations

  • (j) noop finally release → fails cancel_before_reply_clears_correlation on reply_publication_seq_active.
  • B1 old close + old _ensure early-return → fails close_reconnect on publication-timeout assertion.
  • B2 publication_fut.result() instead of exception() → both session_loss_* nodes fail.

Should-fix — internal.py

session_disconnect removed; disconnect/slot checks live in trigger_session_disconnect, reply_publication_seq_active, and reply_publication_pending helpers (33 lines; was 19 pre-B2 harness).

Open in Web Open in Cursor 

Note

Medium Risk
Changes command completion semantics and cross-thread snapshot timing for core area/zone operations; disconnect/cancel cleanup must stay correct to avoid stale state or leaked correlations.

Reviewed by Cursor Bugbot for commit f4ad443. Bugbot is set up for automated code reviews on this repo. Configure here.

cursoragent and others added 5 commits November 15, 2026 15:19
Extend the existing entity status publication wait used by arm and zone
writes so area_get_status and zone_get_all_zones_status do not return until
the receive-thread reply is reflected in the client snapshot. Notify zone
waiters on ZonesStatusBulkUpdated and add regression tests that defer
call_soon_threadsafe to model kernel resolve-before-dispatch ordering.

Co-authored-by: Mitch Mitchell <mitchmitchell1616@msn.com>
Export Elke27PanelApplicationFirmwareUnknownError and
Elke27NightArmFirmwareTooOldError as Elke27InvalidArgument subclasses with
unchanged messages so integrators can catch specific cases without breaking
existing broad handlers.

Co-authored-by: Mitch Mitchell <mitchmitchell1616@msn.com>
Share collection between reconcile and bulk_zone_status_publication_zone_ids
so unknown status characters are not waited on when the handler emits no bulk
event. Add prompt-return regression tests for unknown-char bulk, empty bulk,
and unchanged area_get_status replies.

Co-authored-by: Mitch Mitchell <mitchmitchell1616@msn.com>
Co-authored-by: Mitch Mitchell <mitchmitchell1616@msn.com>
Co-authored-by: Mitch Mitchell <mitchmitchell1616@msn.com>

@mitchmitchell mitchmitchell left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read-side publication waits make sense, and the typed arm-night errors are a nice cleanup for HACS.

Blocking

  • zone.py: the mutate flag + optional now turns a call-site contract into a runtime ValueError that can never fire. Split into a pure "which ids would update" helper and the mutating reconcile with now: float required.

Should fix

  • client.py: per-zone sequential waits re-derive the ids by re-running the parser on the loop thread, and on a missed bulk event each remaining zone logs its own timeout warning. A single bulk-publication seq bumped on ZonesStatusBulkUpdated would avoid both.
  • Duplicated status-char sets in _zone_status_char_recognized / _apply_zone_status_char.

Nits

  • Tests branch on case.id == for setup; put a setup callable on the case dataclass instead.
  • elapsed < 0.5 wall-clock asserts in test_status_read_publication_wait_returns_promptly are the same flake shape as #124; the "no timeout warning logged" assert already proves promptness.

Reason: dead ValueError guard in _collect_bulk_zone_status_updated_zone_ids (zone.py L1054, L1080).

Verdict: CHANGES_REQUESTED @ 89ed15f
joostlek-style review via Claude


Generated by Claude Code

Comment thread src/elke27_lib/handlers/zone.py Resolved
Comment thread src/elke27_lib/handlers/zone.py Resolved
Comment thread src/elke27_lib/client.py Resolved
Comment thread src/elke27_lib/client.py Resolved
Comment thread src/elke27_lib/handlers/zone.py Resolved
Resolve firmware typed-error conflicts using main's definitions and
0.0.5.37 or later wording; drop duplicate unknown-firmware test.

Co-authored-by: Mitch Mitchell <mitchmitchell1616@msn.com>
@mitchmitchell
mitchmitchell marked this pull request as ready for review November 16, 2026 01:38
@mitchmitchell

Copy link
Copy Markdown
Owner Author

re-review @ 3c65289d653e161e39c18104173d8601bb953eaf

Merged main 00d5d4cb (#142) via merge commit 3c65289. Uses main's typed firmware errors (Elke27PanelApplicationFirmwareUnknownError, Elke27NightArmFirmwareTooOldError(message, *, required, actual)) and wording "Arm night requires panel application firmware 0.0.5.37 or later." Duplicate test_arm_night_unknown_application_firmware was dropped; main's test_arm_night_rejected_when_application_firmware_unknown remains. Exactly one definition of each typed error.

Independent verification at this exact tip:

  • pytest: 1412 passed, 42 skipped
  • basedpyright: 0 errors, 0 warnings, 0 notes
  • ruff check/format: clean
  • Guards: PRIVATE_ACCESS_CEILING lowered 96 → 93 on test/test_snapshot_after_arm_bypass.py; NOQA/SLF001 ceilings unchanged; nothing added to LEGACY_PRIVATE_HELPERS or LEGACY_TEST_MODULE_IMPORTS; no new comments, no unreasoned noqa/type: ignore
  • Mutation: disabling the new area_get_status publication wait failed test_async_execute_waits_for_receive_thread_snapshot_publication[area_get_status]; reverted

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-16T01:41:20.648803Z 3c65289 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@mitchmitchell

Copy link
Copy Markdown
Owner Author

@cursor review

@mitchmitchell

Copy link
Copy Markdown
Owner Author

Claude (hourly) — tip verdict @ 3c65289d653e161e39c18104173d8601bb953eaf

Please post the D18 verdict for this exact tip.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3c65289d65

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/elke27_lib/client.py Resolved

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

@mitchmitchell mitchmitchell left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: Changes needed @ 3c65289

First Architect review of #144 (whole PR, at the merge tip with main 00d5d4c). Tests: 1412 passed, 42 skipped; ruff check and format clean; CI and Bugbot green at this tip. The new read-side waits are tested: 5 of 5 mutations failed the tests (no area_get_status wait, no bulk wait loop, no bulk notify, empty predictor, predictor that accepts unknown chars).

B1 (blocker, new in this PR): a bulk zone poll can now release a zone write's publication wait, so a bypass write can return while the snapshot is stale.
_handle_kernel_event now bumps _zone_status_seq[zone_id] for every id in ZonesStatusBulkUpdated.updated_ids (client.py:1882-1884). zone_set_status takes seen_before before it sends, so a bulk poll reply that lands for the same zone between the send and the write's own publication counts as the write's publication, even though it carries the state from before the write. I confirmed this with a probe built on the test_async_execute_waits_for_receive_thread_snapshot_publication harness: start zone_set_status(zone_id=1, bypassed=True), handle one ZonesStatusBulkUpdated(updated_ids=(1,)), then deliver the confirmed set_status reply from a worker thread while its _handle_kernel_event is deferred. At this tip, async_execute returns ok=True with get_snapshot().zones[1].bypassed is False. On main 00d5d4c the same probe returns True. That is the HACS#171 B1 problem again (stale zones reaching the alarm_arm_automatic bypass logic), now coming in through the write path. This is the Codex P2 on client.py:1883.

Fix at the root cause: tie publication to the command's own reply instead of to per-entity counters that any event can bump. Example: the client marks request seq N as published once it has handled the events dispatched from reply N, or once dispatch of reply N finishes, which covers replies that emit no event. async_execute then waits on that one marker for writes and for the new status reads. With that change the bulk predictor (bulk_zone_status_publication_zone_ids, the mutate flag with its ValueError guard, and the per-zone wait loop) is no longer needed, which also settles Claude's inline points on zone.py:1054/1080/1185 and client.py:3476/3480. Add the probe above as a regression test (a bulk poll for the same zone does not release a zone write wait), next to the existing receive-thread cases. A separate bulk counter would be a stopgap: it closes this path but leaves same-zone single-status events able to release a write wait.

No new should-fix items for elke27#50. Developer owns the fix.

Tie async_execute publication waits to command reply seq after
successful snapshot replace; fail waiters on disconnect. Remove
per-zone bulk publication predictors; consolidate zone status char sets.

Co-authored-by: Mitch Mitchell <mitchmitchell1616@msn.com>

@mitchmitchell mitchmitchell left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up since 89ed15f (#142 merge-up plus 20f940e). Moving the wait onto the reply seq fixes the bulk-poll-releases-write-wait race, and the earlier mutate/ValueError blocker is gone. The per-zone re-parse, duplicated status sets and the case.id == branching are addressed too.

Blocking

  • client.py: try / except BaseException: raise / else around _replace_snapshot is a no-op.
  • test_snapshot_failure_does_not_publish_reply_seq monkeypatches _replace_snapshot and asserts on the private _published_reply_seqs.

Should fix

  • _published_reply_seqs is never pruned.
  • bulk_zone_status_reply_will_publish duplicates the reconcile walk; emitting the bulk event unconditionally would remove the predictor.

Reason: no-op try/except in _handle_kernel_event (client.py L2036) and a new test reading private _replace_snapshot / _published_reply_seqs (test_snapshot_after_arm_bypass.py L634, L652).

Verdict: CHANGES_REQUESTED @ 20f940e
joostlek-style review via Claude


Generated by Claude Code

Comment thread src/elke27_lib/client.py Resolved
Comment thread test/test_snapshot_after_arm_bypass.py Resolved
Comment thread src/elke27_lib/client.py Resolved
Comment thread src/elke27_lib/handlers/zone.py Resolved
Rebuild the same-zone bulk vs zone write regression as a receive-thread
case that publishes bulk for zone 1 before the write reply but defers the
write snapshot until after asserting async_execute is still pending.

Co-authored-by: Mitch Mitchell <mitchmitchell1616@msn.com>
@mitchmitchell

Copy link
Copy Markdown
Owner Author

re-review @ 318eda1bb2bfa70ce8072fff5e7e21c7f0e8e470

Independent verification at this exact tip. git diff 20f940e 318eda1 touches only test/test_snapshot_after_arm_bypass.py and test/test_review_guards.py: the B1 probe was rebuilt as receive-thread case same_zone_unrelated_bulk_zone_write, inject_before_first_reply was removed, and PRIVATE_ACCESS_CEILING went 92 → 91.

Architect review 5914149274:

  • B1: one reply-seq marker for writes and status reads (_await_reply_snapshot_publication(seq, …)). bulk_zone_status_publication_zone_ids, _collect_*, the mutate flag and its ValueError guard, and the per-zone wait loop are deleted. No separate bulk counter.
  • Probe: the new case delivers a same-zone (zone 1, status "1") bulk poll at sent["seq"]+100 before the zone_set_status reply, with _handle_kernel_event deferred. Copied the test module onto a 3c65289 worktree and ran it against that old src: failed at assert not task.done() because async_execute had already returned after the bulk event. The same case passed at this tip.
  • Ordering: _complete_reply_publication runs only after a successful _replace_snapshot. Mutation (complete before replace) failed test_snapshot_failure_does_not_publish_reply_seq (seq 42 was published). Reverted.
  • Disconnect: pending markers fail with Elke27DisconnectedError. Mutation (remove _fail_all_reply_publication_waiters) failed with DID NOT RAISE (logged “Timed out waiting for reply seq=1 snapshot publication”). Reverted.
  • Test nits / Claude should-fixes: no case.id setup branches remain; no new wall-clock elapsed asserts; zone status-char frozensets are a single module-level source. The original 6 review threads from that pass remain resolved.

Results at this tip: pytest 1415 passed, 42 skipped; basedpyright 0 errors, 0 warnings, 0 notes; ruff check/format clean; PRIVATE_ACCESS_CEILING 96 → 91 vs main; no new code comments; no unreasoned noqa; nothing added to LEGACY_PRIVATE_HELPERS or LEGACY_TEST_MODULE_IMPORTS.

Note: 4 later Claude threads from review 5914331738 @ 20f940e (8:20 PM CT) are still open — the try/except BaseException/else around _replace_snapshot, the snapshot-failure test patching internals, unbounded _published_reply_seqs, and bulk_zone_status_reply_will_publish as a second payload walk. This tip did not change src, so those remain.

@mitchmitchell

Copy link
Copy Markdown
Owner Author

@cursor review

@mitchmitchell

Copy link
Copy Markdown
Owner Author

Claude (hourly) — tip verdict @ 318eda1bb2bfa70ce8072fff5e7e21c7f0e8e470

Please post the D18 verdict for this exact tip.

@mitchmitchell mitchmitchell left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: Changes needed @ 318eda1

Re-review of 3c65289..318eda1 (20f940e + 318eda1). Only the new code was reviewed.

Previous B1 (bulk poll releases zone write wait): fixed. I ran the new same_zone_unrelated_bulk_zone_write case against the 3c65289 src and it fails there; it passes at this tip. Waiting on the command's own reply seq is the right shape.

Blocking (all in new code)

  • B1: regression. The ack-only zone_set_status → zone_get_status refresh no longer waits for publication. _REPLY_PUBLICATION_COMMANDS leaves out zone_get_status, even though _reply_publication_qualifies handles the ("zone","get_status") route. When the write reply has no BYPASSED, _command_needs_reply_publication_wait returns False for the write. _refresh_snapshot_from_panel_status then runs async_execute("zone_get_status"), and nothing waits on that call. With the receive-thread deferral used by your own probe, async_execute("zone_set_status", bypassed=True) returns ok while get_snapshot().zones[1].bypassed is still False. At 3c65289 the per-zone wait held the write until the refresh was published (probe: returned_early False there, True here). The area path is fine because area_get_status is in the set. A direct async_execute("zone_get_status") is also unwaited. Fix: wait on the reply seq for zone_get_status as well, and add this route to the receive-thread cases (ack-only write + refresh, and a direct read).
  • B2: _published_reply_seqs grows without bound. _complete_reply_publication adds the seq of every qualifying status event: every poll, bootstrap, bulk read and confirmed write reply, including the many that nobody waits on. Nothing ever removes an entry, not even disconnect or close. A long-running HA client therefore leaks one entry per status reply for the life of the process. Seqs come from a monotonic 31-bit counter, so this is a leak and not an early-release bug. Fix at the root: register the awaited seq before the send (or when the reply is correlated), record or complete publication only for registered seqs, and remove the entry when the waiter finishes (published, timed out or disconnected). The global set then goes away.
  • B3: the bulk predictor is still there. In 5914149274 I asked for the bulk predictor to be dropped. bulk_zone_status_reply_will_publish is a second walk of the payload that has to stay in sync with _reconcile_bulk_zone_status, and it runs on the loop against kernel state that the receive thread may already have mutated. The two walks agree today (forcing it to return True or False is caught by the tests). If they ever diverge, the caller either stalls to the command deadline or returns before publication. Fix at the root: completion should not depend on predicting what the handler emits. Either always emit ZonesStatusBulkUpdated for a successful bulk reply (empty updated_ids included), or complete the reply seq once that reply's dispatch has been processed on the loop. Either way, delete the predictor. The second option also covers S5 (area_get_status with area_id < 1 stalls).

Should-fix: #50 (comment) (S1 no-op try/except BaseException/else; S2 snapshot-failure test pins internals; S3 dead get_status route branches; S4 ack-only tests don't assert the wait; S5 area_get_status malformed-reply stall). Claude's four threads: the unbounded set is B2 and the second walk is B3. The no-op try/except and the internals-pinning test are non-blocking in my view (S1/S2).

Checks @ 318eda1: pytest 1415 passed, 42 skipped. ruff check and format are clean. devtools/lint.py (basedpyright + loopcheck) passes. CI is green. No new code comments.
Mutations: 7 of 9 caught: drop the published record; bulk not qualifying; no disconnect fail; drop the wait; complete before replace; predictor forced True; predictor forced False. 2 survived: dropping the qualifies filter (S4) and dropping the area get_status route branch (S3).
Bugbot: running at 318eda1, not yet clean. Claude D18: no verdict at this tip (last was CHANGES_REQUESTED @ 20f940e).

Comment thread src/elke27_lib/client.py Resolved
Comment thread src/elke27_lib/client.py Resolved
Comment thread src/elke27_lib/handlers/zone.py Resolved

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread src/elke27_lib/client.py Resolved
Comment thread src/elke27_lib/client.py Resolved
cursoragent and others added 2 commits November 16, 2026 02:48
Register publication slots only for awaited command seqs, complete them after
a successful snapshot replace (or dispatch fallback on status reads), add
zone_get_status to the publication command set, always emit bulk zone updates,
and extend receive-thread regression tests plus S3/S4 publication qualifiers.

Co-authored-by: Mitch Mitchell <mitchmitchell1616@msn.com>
Rebuild ack-only zone write receive-thread proof; defer dispatch completion
for malformed area reads; drop slots on publication timeout; route panel
delivery through test helper; restore review guard ceilings.

Co-authored-by: Mitch Mitchell <mitchmitchell1616@msn.com>

@mitchmitchell mitchmitchell left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up since 20f940e (318eda1, 6ccdc8c, 0031a4d). Earlier blockers are fixed: the no-op try/except BaseException/else is gone and the _replace_snapshot monkeypatch / _published_reply_seqs test is replaced by a behavioral snapshot-build failure case. _published_reply_seqs growth and the bulk predictor are gone too, and zone_get_status now waits.

Blocking

  • client.py: _dispatch_listener_remove is assigned and never read (dead code).
  • test_reply_publication_timeout_drops_registered_slot asserts on private _reply_publication_by_seq.

Should fix

  • isinstance(self._kernel, E27Kernel) probe; silent if key in COMMANDS filter; unused handled listener arg; duplicated snapshot-build block; fut is None after disconnect returning success (inline).

Nits

  • raise snapshot_replace_failed can be a bare raise (client.py L1714, L2099).
  • deliver_panel_message just moves _on_message behind a helper, so part of the 92 to 88 ceiling drop is the counter not seeing it rather than less private access.

Reason: dead _dispatch_listener_remove (client.py L538/L612) and a new test reading private _reply_publication_by_seq (test_snapshot_after_arm_bypass.py L1244).

Verdict: CHANGES_REQUESTED @ 0031a4d
joostlek-style review via Claude


Generated by Claude Code

Comment thread src/elke27_lib/client.py Resolved
Comment thread test/test_snapshot_after_arm_bypass.py Resolved
Comment thread src/elke27_lib/client.py Resolved
Comment thread src/elke27_lib/client.py Resolved
Comment thread src/elke27_lib/client.py Resolved
Comment thread src/elke27_lib/client.py Resolved
Comment thread src/elke27_lib/client.py Resolved
cursoragent and others added 3 commits November 16, 2026 03:25
…ck test

Register kernel dispatch correlation for reply-publication commands so
dispatch-only get_status replies complete publication without timing out.
Extend the deferred receive-thread harness for ack-only arm/zone paths,
unconfirmed status probes, and dispatch fallback with caplog and budget
control. Re-add deliver_panel_message and inject_unconfirmed_publication_event
in the legacy helper to keep private-access ceilings flat.

Co-authored-by: Mitch Mitchell <mitchmitchell1616@msn.com>
Pair reply-publication slots with kernel dispatch correlation only after a
successful send; release both on every exit path and on disconnect. Set
_connected false on link loss, surface disconnect during publication wait,
dedupe snapshot replace for reply publication, validate command keys at import,
and remove the dispatch listener on close. Restore zone_get_status receive-thread
coverage, ruff-format the snapshot bypass tests, and lower the SLF001 ceiling.

Co-authored-by: Mitch Mitchell <mitchmitchell1616@msn.com>
Co-authored-by: Mitch Mitchell <mitchmitchell1616@msn.com>
…3 pins

Unify panel connect setup via _prepare_panel_connect_listeners on Result and
async connect paths. Add session-loss-while-reply-pending coverage, publication
release-path tests, cancel drop-pending late-reply behavior, and bounded harness
waits. Remove unreachable snapshot-publication slot-gone branch.

Co-authored-by: Mitch Mitchell <mitchmitchell1616@msn.com>

@mitchmitchell mitchmitchell left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up since c29e27a. The blocking point from that round is fixed: the fabricated publication-future test is gone, and test_session_loss_while_reply_pending now reaches the drain through _on_session_disconnected with caplog on the asyncio logger. The should-fix points are addressed too: the recv-thread retry-cleanup test is back, drop_only no longer keys on message text, and the duplicate future.exception() drains are gone. Removing the fut is None and not self._connected branch is fine since _fail_all_reply_publication_waiters leaves the futures in place.

Should fix

  • New late-reply test passes with or without drop_pending_response (inline).
  • Publication release-path test calls three private methods directly (inline).
  • _connect_client_for_fake_panel forces panel.connected again (inline).

Nits

  • Unbounded while not panel.sent spin in the close/connect test (inline).
  • drop_only alias in cancel_command_transport (inline).
  • test_session_loss_while_reply_pending: for zone_id in (1,): is a one-item loop, and the kwargs .get(command, {}) lookup reads better as a second parametrize column.
  • test_public_api_snapshot_publication_error_exported only asserts a docstring exists; importing it in the module-level from elke27_lib import ... block already proves the export.

Reason: No blocking findings.

Verdict: APPROVED @ f55a805
joostlek-style review via Claude


Generated by Claude Code

Comment thread test/test_transport_command_retry.py Resolved
Comment thread test/test_transport_command_retry.py Resolved
Comment thread test/test_snapshot_after_arm_bypass.py Resolved
Comment thread test/test_client_close_resubscribe.py Resolved
Comment thread src/elke27_lib/kernel_loop.py Resolved
… bounds

Format client and transport tests. Disconnect Result connect now asserts events
delivery like the close variant. Add bounded async_connect event pins, T02 spent
budget via past deadline, hold_kernel bounded waits, __all__ export check, and
zone bulk/status coverage for violated B char and updated_count.

Co-authored-by: Mitch Mitchell <mitchmitchell1616@msn.com>

@mitchmitchell mitchmitchell left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up since f55a805. One commit: formatting, plus extra test pins. The only src/ change is the cancel_command_transport call collapsing to one line. The earlier nit on test_public_api_snapshot_publication_error_exported is half addressed (the __all__ check is useful).

Should fix

  • test_disconnect_then_result_connect_receives_events_and_dispatch no longer checks dispatch (inline).
  • getattr on the bulk-status event in test_zone_bulk_status_configured.py (inline).

Nits

  • The three connect-then-event tests are near copies (inline).
  • await asyncio.sleep(0.05) before assert not task.done() (inline).

Reason: No blocking findings.

Verdict: APPROVED @ 20fe6d4
joostlek-style review via Claude


Generated by Claude Code

Comment thread test/test_client_close_resubscribe.py Resolved
Comment thread test/test_client_close_resubscribe.py Resolved
Comment thread test/test_snapshot_after_arm_bypass.py Resolved
cursoragent and others added 2 commits November 16, 2026 14:05
…imeouts.

Use drop_pending_response and pending_responses.drop in the cancel-drop test; drive publication release paths through async_execute with spent-budget and done-future checks; remove manual client.state.panel.connected wiring; bound harness spins with asyncio.timeout; drop the drop_only alias in cancel_command_transport.

Co-authored-by: Mitch Mitchell <mitchmitchell1616@msn.com>
Parametrize async_connect event delivery with a shared drain helper, type zone bulk events directly, and wait on publication registration instead of sleeping in the hold_kernel harness branch.

Co-authored-by: Mitch Mitchell <mitchmitchell1616@msn.com>

@mitchmitchell mitchmitchell left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up since 20fe6d4. Two commits: the release-paths test now goes through async_execute instead of the private _begin/_await/_complete helpers, the cancel-drop test pins pending_responses.drop, and the harness spins are bounded with asyncio.timeout. The earlier should-fixes (dispatch test name, getattr on the bulk event) and the parametrize nit are addressed. The src/ change (inlining drop_only) is fine.

Should fix

  • asyncio.sleep(0.09) against a 0.08 budget in the release-paths test (inline).

Nits

  • The six while not task.done() reply loops now each get their own asyncio.timeout wrap; test/helpers/panel_command_driver.drive_until_done already does this, can the per-file copies reuse it?
  • _drive_until_done in test_snapshot_after_arm_bypass.py uses timeout(10.0) while every other harness uses 2.0; why the difference?

Reason: No blocking findings.

Verdict: APPROVED @ ea3cc05
joostlek-style review via Claude


Generated by Claude Code

Comment thread test/test_transport_command_retry.py Resolved
cursoragent and others added 2 commits November 16, 2026 15:05
Type sent-session wait helpers with a Protocol, pin spent-budget and fut-done release via async_execute with dispatch drop and wait_for checks, and add a dedicated completed-future release test.

Co-authored-by: Mitch Mitchell <mitchmitchell1616@msn.com>
Co-authored-by: Mitch Mitchell <mitchmitchell1616@msn.com>

@mitchmitchell mitchmitchell left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up since ea3cc05. The post-budget asyncio.sleep(0.09) is gone and the test now waits on the spent-publication log; the completed-future path got its own test through async_execute, and _SentSession types the harness helpers. Earlier blocking findings were already fixed at the last review.

Should fix

  • The new _slow_refresh patches the private _status_refresh_after_successful_write and sleeps 0.12s of wall time against the 0.08 budget (inline).

Nits

  • The _spy_drop_dispatch / _keep_match_pending setup is now copied verbatim in both release tests; a small fixture would keep them in sync.
  • _wait_spent_publication_timeout is defined inside the async with block; define it before and just await it under the timeout.
  • await asyncio.sleep(0) right before await task (line 1208) can be omitted.

Reason: No blocking findings.

Verdict: APPROVED @ c5537ce
joostlek-style review via Claude


Generated by Claude Code

Comment thread test/test_transport_command_retry.py Resolved
Remove redundant releases inside _await_reply_snapshot_publication and
rely on the publication_owner finally block. Strip fabricated dispatcher
stubs from transport publication tests; use real spent-budget and dispatch-
only paths with asyncio.timeout bounds.

Co-authored-by: Mitch Mitchell <mitchmitchell1616@msn.com>

@mitchmitchell mitchmitchell left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up since c5537ce. Releasing the publication slot only in the async_execute finally is cleaner: one owner, one release, and the inner _await_reply_snapshot_publication no longer double-drops. The dispatcher/call_soon_threadsafe/wait_for stubs are gone from the release-paths test, so the duplicated spy setup nit from last round is moot.

Should fix

  • The folded-in "fast" case still runs under the patched 0.08s budget and the 0.12s _slow_refresh, so it takes the spent-budget branch again; the fut.done() path lost its coverage (inline).

Nits

  • wire_client_loop is the same set_private pair as L731 and L1010; those tests could use the helper too.
  • The wait loop at L1099 can be a single condition (inline).

Reason: No blocking findings.

Verdict: APPROVED @ f7f75ca
joostlek-style review via Claude


Generated by Claude Code

Comment thread test/test_transport_command_retry.py Resolved
Comment thread test/test_transport_command_retry.py Resolved
cursoragent and others added 2 commits November 16, 2026 16:55
… publicly

Release clears only the publication future slot; dispatcher cleanup is already
handled by match_pending on reply, cancel_command_transport, and _abort_requests.
Drop wire_client_loop set_private; drive release-path tests via connect_panel.

Co-authored-by: Mitch Mitchell <mitchmitchell1616@msn.com>
…ort test

Co-authored-by: Mitch Mitchell <mitchmitchell1616@msn.com>

@mitchmitchell mitchmitchell left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up since f7f75ca. Driving the release-path test through connect_panel and restoring the real budget for the fast case is better; the fut.done() path is covered again, and the wire_client_loop set_private nit is gone.

Blocking

  • Dropping drop_dispatch_correlation leaves the send-failure path without cleanup: _begin_reply_publication_command registers the correlation before send_request_with_seq, and when that raises (no session / session not ACTIVE) only pending_responses is dropped. The dispatcher is never recreated, so the entry stays forever (inline).

Should fix

  • The release-path assertions went from pending_count() == 0 to match_pending(seq); a leaked correlation on another seq no longer fails them (inline).

Reason: Dispatcher correlation leaks when send_request_with_seq raises after _begin_reply_publication_command.

Verdict: CHANGES_REQUESTED @ 9d09a59
joostlek-style review via Claude


Generated by Claude Code

Comment thread src/elke27_lib/client.py Resolved
Comment thread test/test_transport_command_retry.py Resolved
cursoragent and others added 3 commits November 16, 2026 17:35
…ct flake

Re-add drop_dispatch_correlation in the single async_execute finally release
path so send failures after register_dispatch_correlation cannot leak pending
correlation. Add behavioral async_execute coverage for inactive and missing
session. Drive disconnect_before_reply through trigger_session_disconnect;
spent publication budget via public arm refresh timing.

Co-authored-by: Mitch Mitchell <mitchmitchell1616@msn.com>
Co-authored-by: Mitch Mitchell <mitchmitchell1616@msn.com>
…transport

Remove the client_execute_coverage regression that bumped that file's ceiling.
Add send-failure/async_execute correlation cleanup via async_connect, panel.close,
and client.close only, asserting dispatcher pending does not grow. Lower transport
PRIVATE_ACCESS ceiling to measured 83.

Co-authored-by: Mitch Mitchell <mitchmitchell1616@msn.com>

@mitchmitchell mitchmitchell left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up since 9d09a59. Restoring drop_dispatch_correlation in _release_reply_publication_command fixes the earlier blocker: the finally covers the send_request_with_seq failure path, and test_async_execute_send_failure_clears_dispatch_correlation fails against the 9d09a59 client and passes here (checked locally, 3/3 runs green).

Should fix

  • The spent-budget case now accepts either the publication timeout or "follow-up status read failed", so it can pass without ever hitting the publication-timeout path (inline).
  • await asyncio.sleep(0.12) ties the case to wall-clock against a 0.08 s budget (inline).

Nits

  • Duplicate sent_before line; the two send-failure cases could be a parametrize (inline).

Reason: No blocking findings; the dispatcher correlation leak is fixed.

Verdict: APPROVED @ b25fb58
joostlek-style review via Claude


Generated by Claude Code

Comment thread test/test_transport_command_retry.py Resolved
Comment thread test/test_transport_command_retry.py Resolved
Comment thread test/test_transport_command_retry.py Resolved
Comment thread test/test_transport_command_retry.py Resolved
cursoragent and others added 2 commits November 16, 2026 18:31
Restore f7f75ca-style budget burn via status refresh delay, clear caplog
between halves, and require the publication-timeout warning only. Transport
PRIVATE_ACCESS ceiling 84 (measured).

Co-authored-by: Mitch Mitchell <mitchmitchell1616@msn.com>
Use pytest.mark.parametrize for panel.close() versus client.close() while
keeping dispatcher pending_count stable after failed async_execute.

Co-authored-by: Mitch Mitchell <mitchmitchell1616@msn.com>
@mitchmitchell

Copy link
Copy Markdown
Owner Author

re-review @ f4ad443bb81ddef56230886cc92572fddf340388

Independent verification of this tip. Diff vs 27ad73e is test-only: test_async_execute_send_failure_clears_dispatch_correlation is parametrized over panel.close() (inactive_session) and client.close() (no_session). Src is unchanged from 27ad73e / b25fb58, so those production results carry over. The four previously unresolved review threads are resolved (GraphQL: 0 unresolved / 82 resolved / hasNextPage false, paged).

Production fixes that carry over:

  • Release site: _release_reply_publication_command calls idempotent drop_dispatch_correlation (dispatcher.drop_pending). Single release from async_execute finally.
  • Leak: begin still registers the dispatcher entry before send. If send_request_with_seq raises because the session is inactive or missing, _release now drops that entry. Probe on b25fb58 (src unchanged): pending stays 0 after exactly one release on happy, spent-budget, session-loss, cancel, timeout, abort, send-failure, session-inactive, and no-session paths.
  • Publication wait: spent-budget early return is uniquely pinned by test_reply_snapshot_publication_release_paths (T02 killed on the publication-timeout log at 27ad73e). Disconnect-before-reply uses trigger_session_disconnect with no receive thread (50× clean at 27ad73e / b25fb58). Real T05, B1g, and S10 were full-suite kills under 240s; K07/R02 were behavioral kills via this send-failure test.

This tip:

  • Fresh clone, UV_LINK_MODE=copy. basedpyright 0, whole-tree ruff check and format, loopcheck 0, review guards 10 passed.
  • Measured ceilings: PRIVATE_ACCESS 1830 (transport 84, at most 85), GETATTR 56, NOQA 35, every per-file unchanged vs 27ad73e. LEGACY lists and pyproject.toml match main. test/helpers/internal.py unchanged.
  • 3 full suites: 1449 passed, 9 skipped, 33 deselected.
  • K07 (drop_dispatch_correlation no-op) and R02 (release without drop) each fail both parametrized cases and only those.
  • Fail-on-old vs 9d09a59 src: both cases fail (inactive_session 3==2, no_session 1==0).
  • 50× under load: 0/50 on both send-failure cases.

@mitchmitchell

Copy link
Copy Markdown
Owner Author

@cursor review

@mitchmitchell

Copy link
Copy Markdown
Owner Author

Claude (hourly) — tip verdict @ f4ad443bb81ddef56230886cc92572fddf340388

Please post the D18 verdict for this exact tip.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit f4ad443. Configure here.

@mitchmitchell mitchmitchell left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: Approved (code) @ f4ad443

Re-review of 8f3acaf..f4ad443. Last round's B1 is fixed at the root. close() no longer detaches the kernel subscription, so ConnectionStateChanged(connected=False, reason="closed") reaches _handle_kernel_event. Subscribers get the close, events() ends, and in-flight publication waits fail with Elke27DisconnectedError. Result connect() now re-ensures the listener through _prepare_panel_connect_listeners(), just as async_connect does. The three regression cases I asked for exist in test_client_close_resubscribe.py. Moving the publication release into the single async_execute finally, with an idempotent drop_dispatch_correlation, gives one owner and covers the send-failure path. The cancel path now drops the pending response instead of failing it, and that closes the old never-retrieved Future (S3 on #50). The f4ad443 delta is test-only. It pins the spent-budget case to the publication-timeout log, which addresses Claude's should-fix at b25fb58, and it parametrizes the send-failure case.

No new blockers. The two should-fix items (an unpinned _connected = True in Result connect(), and the spent-budget return skipping a done publication future) are on #50: #50 (comment)

Checks (fresh clone): 1449 passed / 42 skipped; ruff check and format on src/test, devtools/lint.py clean; CI green. Mutations: 4/5 caught. Re-adding the detach in close() fails 4 cases, dropping the prepare from connect() fails 2, release without the dispatch drop fails 2, and removing the disconnect fail-waiters fails 4. Removing _connected = True from connect() survives the full suite (S1). Bugbot: clean at f4ad443 (review 5925719970, no issues). D18: Claude's latest approval is at b25fb58. Merging still needs Claude Verdict: APPROVED @ f4ad443bb81d at this head.

@mitchmitchell mitchmitchell left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up since b25fb58. The spent-budget case now only accepts the publication-timeout log, so it pins the remaining <= 0 branch of _await_reply_snapshot_publication, and the send-failure cases are parametrized. Both earlier points are addressed.

Should fix

  • The new seam patches a private client method and bumps the private-access ratchet (inline).

Reason: No blocking findings.

Verdict: APPROVED @ f4ad443
joostlek-style review via Claude


Generated by Claude Code

await asyncio.sleep(0.12)

monkeypatch.setattr(
client, "_status_refresh_after_successful_write", _burn_budget_before_publication

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to patch _status_refresh_after_successful_write here? Holding the publication (like hold_dispatch_publication does elsewhere) or shrinking the budget further would hit the same branch without another private access in the ratchet.


Generated by Claude Code

@mitchmitchell mitchmitchell left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Decision 55 Approved (offline only) @ f4ad443

Offline only, with no panel, no TEST HA and no hardware. I used a fresh clone at this exact head and a fresh Python 3.14.8 venv (ran uv cache clean elke27 first), and compared against main 00d5d4cb. All four blockers from my 8f3acaf draft are fixed and each is pinned by a behavior test.

Gates (all green)

  • pytest -k "not test_live_e27_": 1449 passed, 9 skipped (main: 1407, so +42). I ran it 3 times in a row and got 1449 each time, with no flakes.
  • basedpyright: 0 errors. devtools/lint.py: passed. ruff check --no-fix src test and ruff format --check: clean (251 files). Repo-wide ruff check shows the same 6 examples/ findings as main.
  • test_review_guards.py: 10 passed. No new code comments; Elke27SnapshotPublicationError now has a docstring.

Mutation testing: 90 mutants, 59 killed, 31 survived (25 equivalent, 6 small gaps), 0 timeouts or hangs
I reused the 8f3acaf mutant set where the code still exists, adapted 4 mutants to the new code, and added 18 for the guards and branches that are new or changed between 8f3acaf and f4ad443. A canary and a no-op control confirmed the mutated copies were the code under test, and every kill is a failing behavior test. Seven 8f3 mutants are moot because their code is gone: B1a, B2d, B2g, T01, T04, T07, R05. The release now happens in a single place, the async_execute finally; removing that release (R04) or the pop (R03) is killed.

My 8f3 blockers

  1. One attach path for every connect: fixed. Both connect() and async_connect() call _prepare_panel_connect_listeners(), and close() no longer detaches. test_close_then_result_connect_receives_events_and_dispatch and test_disconnect_then_result_connect_receives_events kill the mutant that drops the call from connect(). Dropping it from async_connect, which hung at 8f3, now fails 71 tests cleanly. Re-adding the detach in close(), always re-subscribing, and skipping the detach before re-subscribing are each killed by test_close_and_repeated_connect_prepare_single_dispatch_listener.
    • Leak check: after 5 rounds of close → connect() → disconnect() → connect(), plus 5 rounds of async_disconnect() and re-prepare, the kernel still has exactly 1 subscriber and 1 dispatch listener. No leak and no duplicate.
  2. B2 reply-pending session loss: fixed. test_session_loss_while_reply_pending covers 3 routes, and test_async_execute_connection_lost_maps_to_disconnected_result is also added. Removing the ConnectionLost → Elke27DisconnectedError mapping, dropping the publication_fut.exception() retrieval, and dropping the bulk route from the wait keys are all killed now. The bulk-key mutant was only racy at 8f3.
  3. S1: fixed. (a) Disabling the spent-budget branch is killed via the publication-timeout log in test_reply_snapshot_publication_release_paths. Its release mutant is moot because of the single release. (c) The listener lifecycle is now "one listener for the client's lifetime", pinned as in item 1. (b) The unreachable Elke27DisconnectedError raise is deleted.
  4. Harness timeouts: fixed. The B1 harness wait is _wait_until_deferred_nonempty, and the cancel-case waits are under asyncio.timeout(...). I scanned the three test files and found no unbounded waits left: the remaining loops either pop a finite list or have a fixed range(N).

Developer's claims, checked independently

  • Confirmed: 1449 passed ×3.
  • Confirmed killed: B1g (as the async_connect prepare mutant), S10, T02, T05, K07, R02, E01, Z06, Z07 and Z08. K07 and R02 are now killed by the send-failure correlation test, a behavior test that replaces the spy.
  • Confirmed moot: T01, T07 and B2g.
  • Confirmed equivalent: B2m (the finally consume is redundant with it), and B1c, B1d, K05 and B1_orig_combo (_detach_kernel_subscription and the kernel remove() never run any more).
  • Confirmed surviving: D07 and K04.
  • Not in the claim, but also surviving: the equivalents and gaps listed below.

Other equivalent survivors: the other halves of the detach logic (B1b, B1e, B1f, B1i), ensure inside _prepare_panel_connect_listeners (it's a no-op now, since the subscription is never removed), publication-first ordering and wait-set variants (B2i, B2k, B2l), the in-finally consume (N08, redundant with the in-loop consume), retrieving the reply future in finally (N09; result() already retrieves it on every path), no cancel of a popped future, correlation-after-send (S2 ordering can't be observed), the dispatch-only future guard, the _event_loop is None guard, the import-time registry check, elif → else for bypassed, the kernel listener classification and seq>0 filters, the dispatcher drop in _finish_cancel (the finally release also drops), and the residual if fut is None: return (unreachable).

Non-blocking gaps (track or fold into a later cleanup)

  • D07 (closed-loop RuntimeError suppress) and K04 (listener-failure isolation) are untested.
  • connect() now sets _connected = True. That's a behavior change to the Result-style API with no test pinning it.
  • drop_pending_response=True on the TimeoutError-while-cancelling path (client.py L1021-1025) is never run.
  • Skipping _signal_sent_event in the cancel path survives. The line was moved, not added, so this gap probably also exists on main.
  • Dropping __cause__ in _disconnected_from_connection_lost survives. It's cosmetic.
  • Root-cause cleanup: with close() no longer detaching, _detach_kernel_subscription, the dispatch-listener remove handle and the ensure call inside _prepare_panel_connect_listeners are dead code (client.py L627-631 and kernel L1262-1263 are never run). Either delete them, or make the "one subscription for the client's lifetime" contract explicit in the _ensure_kernel_subscription docstring, so a future change doesn't bring back the 8f3 close/reconnect bug. The if fut is None: return in _await_reply_snapshot_publication is unreachable and can go too.
  • Duplicate consume: _consume_done_publication_future() runs both after the wait loop and in finally. One call site is enough.

Logs: /workspace/d55q/e144-f4a-logs on the Tester box.

@mitchmitchell
mitchmitchell merged commit 7740113 into main Nov 16, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants