Repository navigation
Stop the deploy sweep paying for a web search it discards - #242
Merged
Merged
Conversation
The sweep calls ainvoke without enable_postprocess, so the postprocess node ran a Tavily search for each of its fifteen questions. The sweep reads `answer` and nothing else, so every one of those results was thrown away -- and this runs after each beta deploy, which is where the cost repeats. The same defect as the answer endpoint had, in the other caller. Found while checking, before deploying beta, whether the post-deploy sweep would spend money on searches nobody reads. It would have, so that deploy ran with --skip-sweep. Each question also waited for the search before completing, which is why the sweep's per-question timings were slower than the endpoint's on the same questions. Pinned by a test, because nothing fails when this regresses: the cost is invisible from the outside. The test also guards its own assumption -- if the sweep ever starts reading additional_content, the guard is no longer the right one and says so. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Found while checking, before deploying beta, whether the post-deploy sweep would spend money on searches nobody reads. It would have — so that deploy ran with
--skip-sweep.The defect
answer_sweepcallsainvokewithoutenable_postprocess, so it defaults toTrueand the postprocess node runs a Tavily web search for each of the fifteen questions. The sweep readsanswerand nothing else, so every result is discarded.This is the same defect the answer endpoint had (fixed in #237), surviving in the other caller. It runs after every beta deploy, which is where the cost repeats.
It also made each question slower — the sweep waited for a web search before completing — which is why its per-question timings ran slower than the endpoint's on the same questions.
Why a test rather than just the fix
Nothing fails when this regresses. The cost is invisible from the outside: the sweep still passes, the answers are still right, and the only signal is a bill. So it is pinned, and the test guards its own assumption — if the sweep ever starts reading
additional_content, the guard is no longer the right one and says so.Mutation-checked: flipping the flag back to
Truefails the test.Related
Beta is now running the answer endpoint (tag
480db64), verified end to end through the public URL. Two things worth recording from that:/chat/:python-httpxis blocked while curl and browser UAs pass. That is the site's automation blocking, not this service, but it will bite the website's server-side proxy if their HTTP client sends a default library UA. They have been told.CI-equivalent locally: ruff, format, mypy (129 files), full suite with no API keys set.
🤖 Generated with Claude Code