Adversarial review: the T023 signal was only tested against a stub graph

[adamjohnwright]

Follow-up review of #270, which had sabotage verification but no dedicated pass.

The gap

Every sweep test uses a StubGraph returning a plain dict, so nothing exercised the step that actually matters: whether live_tool_failed survives LangGraph's state merge between the node that sets it and the ainvoke the sweep reads.

Checked on the real graph with a tool patched to raise:

live, tool raises    live_tool_failed=True   answer="I could not find the specific species represented in React..."
ordinary retrieval   live_tool_failed=None   answer="In the context of Alzheimer's disease, CDK5, particularly..."

It survives. And the live answer is exactly the prose that used to trigger a retry — so the signal is what separates them, demonstrated on the real path rather than a stand-in.

The finding

An omission that is correct and reads like an oversight: the no-tools fallback does not set the flag, and should not. get_mcp_tools remembers a failed start, so every later call returns None too and a retry could never succeed — marking it transient would buy the sweep a second attempt guaranteed to fail. It is a persistent condition and the gate should fail loudly (Principle IV).

Now stated where the code is and pinned by a test, because the next person to notice the gap will otherwise close it.

One of my tests was wrong, not the code

It asserted the fallback leaves the selection at None, when the fallback sets [] explicitly. Both mean every collection, so it was asserting a representation rather than the meaning. It now asserts what resolve_collections makes of it.

571 passed, 1 skipped; mypy over 148 files, ruff clean.

🤖 Generated with Claude Code