Skip to content

Slice 126: a retraction, and nothing about an approval is discarded in silence - #101

Merged
HackTuah merged 2 commits into
mainfrom
slice/126-approval-resume-race
Sep 26, 2026
Merged

HackTuah merged 2 commits into
mainfrom
slice/126-approval-resume-race

Conversation

@HackTuah

@HackTuah HackTuah commented Sep 26, 2026 •

Copy link
Copy Markdown
Member

This slice is a retraction. The defect it was opened for does not exist. The scope here is a correction, not a handshake rewrite, and deliberately does not invent a product bug to fill itself.

What I got wrong

The approval handshake is correct. A decision arriving while the Session is in tool_wait is postponed and redelivered on the next state change, exactly as the comment in session.ex claims. A decision made straight through Permissions.decide_request/3, with no LiveView in the loop, is applied. The invariant already held.

Both reproductions are withdrawn.

The suite one was my own test bug. It ended with collect(id, &match?({:state, :idle}, &1), 15_000). collect/3 reads the mailbox, and an {:state, :idle} broadcast from before the message was sent was already sitting in it, so it returned at once and every assertion ran mid-turn. The tell was in the output all along: Finished in 0.2 seconds for a test nominally waiting fifteen.

The live one was scripts/dev_demo_approval.sh, which I wrote the previous day for the README assets. The trace is what settles it:

TRACE DROPPED {:approval, :requested, id="01a0de1a-..."} in state :tool_wait
TRACE postponed :decided in tool_wait
                                       <- and nothing further, ever

tools_done never arrives. The Session is not losing the decision; it postpones it correctly and is still waiting on a tool task that never answers. So the question was never "why is the decision lost" but "why does the tool task never return".

The script stripped pool: Ecto.Adapters.SQL.Sandbox from Trinity.Repo and did nothing for Trinity.Repo.Receipts. Outside the suite that pool sits in auto mode and, as config/test.exs warns in its own comment about the other repo, "the first long-lived process to query holds a connection until it exits." Effects.Runner.execute/3 receipts a decision before it dispatches, which is what makes "nothing fails open" true, so every Receipts.append/2 queued behind that holder and the tool task blocked after the approval row existed and the card was painted but before it could answer the Session.

Hence consumed_at: nil, a chain carrying the ask and not the allow, and a visible error: the tool timed out. Every symptom I reported was real. The cause was mine. The intermittency was the sandbox connection sometimes happening to be free.

What lands

The harness fix, which is the actual defect

unfixed fixed
status after the card renders tool_wait at 14s and counting approval_wait at t=0s
turn completes no yes
repeat drives stalled intermittently 5 of 5
chain ask only ask, allow, effect admit, effect done

Three changes: :pool deleted from both repos, receipts pointed at their own database rather than the test suite's, and no apostrophes inside the -e block. That last is not cosmetic: one apostrophe terminates the single-quoted string and the script dies producing no output at all.

One change under lib/, red first

The catch-all handle_event(:info, {:approval, _, _}, _state, _data) discarded any unmatched approval event without a word. A trace shows :requested discarded in tool_wait on every single turn.

  • :requested stays routine and is deliberately not given a second path into the approval flow. Which calls are held is learned from tools_done, in one place; learning it twice is how a call gets held twice. Logged at debug, dropped.
  • Anything else is an answer with no held call to apply it to, the one shape that can lose a decision. Logged as a warning and receipted as outcome=dropped, naming the state it arrived in.

The receipt is written off the session's task supervisor, never inline, because blocking the Session on a Receipts.append/2 call is the precise shape of the stall this slice was opened to explain.

Red first, against the old clause:

  1) test AC6: an approval event with no held call to apply it to is logged and receipted, never
     dropped in silence
     the drop was not logged. A decision the owner made went nowhere and said nothing.
Result: 3/4 passed

With the clause in place: Result: 4 passed.

Three LiveView tests as regression cover, and they are NOT red-then-green

Said plainly because it would be easy to present them as a fix. All three pass against unmodified lib/. There is no application change behind them. PROOF.md states this in those words.

They are worth keeping because every approval test in the tree synchronises on {:state, :approval_wait} before clicking, which no real caller can do: the card renders from the gate's :requested broadcast, emitted from inside the tool task before the Session has returned from start_tools. The ordering a browser actually produces had never been exercised. It is now, waiting on the painted card and a real deadline, never on collect(:idle) or collect(:approval_wait).

AC5 is the one that proves the invariant already held: it decides through the gate with no view in the loop at the first observable instant, and it passes.

A flake I introduced and removed: 3 failures in 10 on checking for the card before the LiveView had re-rendered, because the test subscribes to the same topic and can win that race. 12 of 12 after waiting for the card.

mix gate ; echo "EXIT=$?"
prod_check: PASS
3607 mods/funs, found no issues.
trinity.names: OK over 648 tracked files
Result: 825 passed (14 properties, 811 tests), 18 excluded
trinity.coverage: 042 80.31% vs 005 80.31%: OK
EXIT=0

For the reviewer

Read the retraction in NOTES.md first. The main output of this slice is that I was wrong, and you should be able to check that rather than take it. The only lib/ change is the discard clause; worth confirming that logging and receipting a dropped decision is what you want rather than raising or refusing.

… consumed

Three failing tests, committed before any fix, by name:

  AC1: an allow clicked on the requested broadcast, with no wait for
       approval_wait, consumes the row and runs the call
       -> consumed_at is nil after status "allowed", decision "once"

  AC3/AC4: the chain reads ask then allow, both on the same scope, and neither
       is a timeout
       -> the chain carries ["ask"] and nothing else. The owner answered and
          the chain does not carry the answer.

  AC5: at the instant the requested broadcast is observable, the session can
       already accept a decision
       -> decided straight through the gate, no LiveView involved at all, and
          the grant is still not applied

The difference from every approval test that passes is one line those tests have
and a browser does not:

  collect(id, &match?({:state, :approval_wait}, &1))

The card is rendered from the gate's :requested broadcast, which is emitted from
inside the tool task before the Session has returned from start_tools. So the
suite has only ever exercised the ordering that works, and the synchronisation
it relies on is not something any real caller can perform.

AC5 is the one that rules out "the LiveView should wait": it decides through
Permissions.decide_request/3 with no view in the loop and fails the same way.

Note on reading the failing output: these runs also log a DBConnection ownership
crash from the tool task. That is a consequence, not the defect. The assertions
on status and decision pass before consumed_at fails, so the decision is
recorded and never consumed; the ownership report is in-flight work colliding
with sandbox teardown after the assertion has already failed. The existing
approval_live_test.exs produces none of it and stays green.

Signed-off-by: Ayla Croft <aylacroft@proton.me>
… in silence)

This slice is a retraction. The defect it was opened for does not exist.

The approval handshake is correct. A decision arriving while the Session is in
tool_wait is postponed and redelivered on the next state change, exactly as the
comment in session.ex claims, and a decision made straight through
Permissions.decide_request/3, with no LiveView in the loop, is applied. The
owner's invariant already held.

Both reproductions I reported are withdrawn.

The suite one was my own test bug. It ended with

    collect(id, &match?({:state, :idle}, &1), 15_000)

and collect/3 reads the mailbox, where an {:state, :idle} broadcast from before
the message was sent was already waiting. It returned at once and every
assertion ran mid-turn. The tell was in the output all along: "Finished in 0.2
seconds" for a test nominally waiting fifteen.

The live one was scripts/dev_demo_approval.sh, which I wrote the previous day
for the README assets. It stripped the sandbox pool from Trinity.Repo and did
nothing for Trinity.Repo.Receipts, so outside the suite that pool sat in auto
mode and, as config/test.exs warns in its own comment about the other repo, the
first long-lived process to query held the connection until it exited.
Effects.Runner.execute/3 receipts a decision before it dispatches, which is what
makes "nothing fails open" true, so every Receipts.append/2 queued behind that
holder and the tool task blocked after the approval row existed and the card was
painted but before it could answer the Session. Hence consumed_at nil, a chain
carrying the ask and not the allow, and a visible "error: the tool timed out".
Every symptom was real and the cause was mine. The intermittency was the sandbox
connection sometimes being free.

What lands:

- scripts/dev_demo_approval.sh: the sandbox pool stripped from both repos, the
  receipts database pointed at its own file rather than the test suite's, and no
  apostrophes inside the -e block. That last one is not cosmetic: a single
  apostrophe terminates the quoted string and the script dies with no output.
  Browser drives go from stalling in tool_wait at 14s and counting to reaching
  approval_wait at t=0s and completing 5 of 5, with the chain reading ask,
  allow, effect admit, effect done.

- One change under lib/, and only one, because the instruction was explicit that
  no product bug be invented to fill the slice. The catch-all for
  {:approval, _, _} discarded any unmatched approval event without a word, and a
  trace showed :requested discarded in tool_wait on every single turn. A
  :requested stays routine and is deliberately not given a second path into the
  approval flow. Anything else is an answer with no held call to apply it to,
  which is the one shape that can lose a decision, so it is logged and receipted
  as outcome=dropped. The receipt is written off the session's task supervisor,
  never inline, because blocking the Session on a receipt write is the precise
  shape of the stall this slice was opened to explain. Red first: AC6 fails
  against the old clause at 3/4 and passes at 4/4.

- Three LiveView tests as regression cover. They pass against unmodified lib/
  and PROOF.md says so rather than dressing them as a fix. They exist because
  every approval test in the tree synchronises on {:state, :approval_wait}
  before clicking, which no real caller can do, so the ordering a browser
  actually produces had never been exercised.

Signed-off-by: Ayla Croft <aylacroft@proton.me>
@HackTuah
HackTuah merged commit 3d29043 into main Sep 26, 2026
8 checks passed
@HackTuah
HackTuah deleted the slice/126-approval-resume-race branch September 26, 2026 15:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant