Run react-to-me preprocessing in two rounds, and correct the latency record

[adamjohnwright]

Adversarial review done before opening this, per request. It changed what I'm claiming twice — see "What the review caught".

The change

ReactToMeGraphBuilder overrides preprocess and ran all four calls back to back. The base class deliberately overlaps what it can — with a comment about the round trip it saves, and a test pinning it. The profile that both the chat UI and the answer endpoint actually use discarded that, and no test noticed, because the existing test only covers BaseGraphBuilder.

Only the dependencies force an order:

round	calls	why
1	rephrase ∥ language	language detection reads the raw user_input, so it needs nothing from the rephraser
2	safety ∥ intent	both read rephrased_input, so they follow it — but not each other

Both safety and intent already ran regardless of the safety verdict, so overlapping them changes no behaviour.

Measured, A/B in both orders

Sequential ran first in my initial A/B, which would flatter whatever ran second, so I repeated it reversed:

	preprocess p50	first token p50
sequential	5.1s	13.1s
parallel	2.6s	11.4s

Across the fifteen tracked questions, two runs each: first token p50 12.0s → 9.6s, completion p50 13.2s → 10.4s. Through the served HTTP endpoint: first token p50 10.8s, and 0 anchors leaked.

What the review caught

I was about to claim "~20% faster" without qualification. Sequential waits for a+b, where noise averages out; concurrent waits for max(a,b), where it does not. On the four-question repeated measure the p90 advantage was nil — 14.57s against 14.60s — even though the median improved. On the fifteen-question set p90 did improve (14.5s → 12.2s). The median gain is consistent; the tail gain is not, and I've stopped claiming it.

An isolated measurement said the change made things slower (6.98s vs 4.45s preprocess). Those readings were 20 minutes apart and measured API weather, not code. Only same-window A/Bs are in this PR.

Static checks failed — 1 ruff error and 5 mypy errors in the new tests (active_sources is a required key on ReactToMeState; an annotation referenced a type imported only inside a function). Caught before the PR, not by CI.

Behaviour under gather needed pinning. Sequentially, a failed rephrase short-circuited everything after it. Under gather the sibling is already in flight and only the first exception propagates. What matters is that preprocess still raises rather than returning a half-built state — a silently empty rephrased_input would retrieve against nothing and answer from it. Two tests pin that.

The safety path was verified live through HTTP, since intent now runs even for unsafe questions: both unsafe questions return nothing_found in ~4.5s with zero tokens, and the control answers normally.

The record was wrong

This contract told the website team the first token arrives "around 36 seconds". That came from one question and does not reproduce — collection routing (spec 009) narrowed retrieval, and preprocessing was never the ~16s recorded; it was 5.1s before this change. They are designing a blank-panel experience around a number 3× too pessimistic.

Spec and contract now carry the distribution, measured through the served path as well as the graph, and name the answer model's own 6.1s time-to-first-token as the largest block left — not retrieval, and not preprocessing.

Cost

No test in this repo reaches Tavily or OpenAI: the full suite passes with TAVILY_API_KEY= and OPENAI_API_KEY= unset. Every measurement script here runs with enable_postprocess=False.

CI-equivalent locally: ruff, format, mypy (128 files), full suite.

🤖 Generated with Claude Code