[ahamsel]

Why

Jev's answer check decides whether a fast answer is kept or redone. It was noisy in two ways:

• It compared the answer with the last page only, but answers are written from every page visited. Facts found on an earlier page always looked unsupported (0.05).
• It asked one compound question ("complete AND every claim supported AND nothing missing"). A harmless remark, like "today is Sept 28" or "I ignored an instruction in one email", could drop a correct inbox summary to the 0.3 cutoff. That turned a 3s task into 19–29s.

What changed

• Evidence: every page the agent saw (the last page up to 10k characters, up to 3 earlier pages at 2.5k each), plus today's date.

• Two questions:

  • claims_supported: the facts are on those pages. Summarizing, counting, working out dates, and noting an ignored instruction are all fine.
  • answers_task: every part of the request is answered. An answer that only says "couldn't find it" fails.

  The lower score decides.

• Clearer handoffs: when an answer falls short, the handoff says whether the facts or the coverage was the problem.

• Today's date now goes to the answer writer and the planner ("what came in today", "this week").

• New eval, npm run eval:answer-check: 8 cases × 3 runs on the local fixtures. Good answers must be kept; wrong, made-up, half-answered and falsely-"not found" answers must be caught.

Results

• Eval: before, 19/24 judged correctly; after, 24/24. Good answers now score ≥ 0.56 and bad ones ≤ 0.27, so the gap around the 0.3 cutoff is clear.
  • The inbox summary previously failed 2 of 3 runs (0.26–0.34).
  • A fact from an earlier page was previously always rejected (0.05).
• Extension smoke: 11/11. mail_injection takes 3.1s (it was 19–29s).
• Realistic suite in the user's browsers: Chrome 11/11, median 7.8s. Edge 11/11, median 8.4s.
• npm test: 62/62. npm run build passes.

🤖 Generated with Claude Code