fix(budget): bill metered LLM usage instead of a flat input-only estimate - #662
Closed
Mr-Neutr0n wants to merge 1 commit into
Closed
fix(budget): bill metered LLM usage instead of a flat input-only estimate#662Mr-Neutr0n wants to merge 1 commit into
Mr-Neutr0n wants to merge 1 commit into
Conversation
…mate The main turn path recorded one estimateCostUsd(inputTokens) per turn, pricing every model at DEFAULT_AGENT_INPUT_USD_PER_MTOK and ignoring output/cache tokens entirely, while the per-step metered usage that each harness already reports through recordLlmRequest (input, output, cacheRead, cacheWrite, provider-computed costUsd) never reached the budget tracker. Budget accounting for turns now comes from that metered usage via costFromUsage(): it prefers the provider-reported costUsd and falls back to a total-token estimate at the default rate when a step reports tokens but no cost. Steps with no reportable usage bill zero rather than a guessed figure. Fixes #586 This change was prepared with AI assistance under human direction and review.
Collaborator
|
Thanks for this, @Mr-Neutr0n — and sorry for the slow response. We're closing this PR. Per We don't want your idea to get lost, so we've captured it in #748 to triage on our side. Please feel free to open an |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #586
Summary
Turn-level budget accounting now uses the metered usage each harness already reports per LLM request instead of a single flat, input-only estimate.
Root cause
recordModelCallfires once per turn with an estimated input-token count only, and the budget recordedestimateCostUsd(rec.inputTokens)from it: every model priced atDEFAULT_AGENT_INPUT_USD_PER_MTOK(5), no output or cache tokens. The measured numbers existed on a second callback the budget never saw:recordLlmRequestreceivesusage(input,output,cacheRead,cacheWrite, provider-computedcostUsd) for every step.What changed
costFromUsage()insrc/ratelimit/budget.ts: prefers the provider-reportedcostUsd, falls back to(input+output+cacheRead+cacheWrite)at the default rate when tokens are reported without cost, and returns 0 when nothing is reportable.recordLlmRequesthandler; the once-per-turn estimate is removed there so steps are not double-counted. Detect, compaction, and security-screen paths keep their existing estimates (they have no metered counterpart).Verification
test/budget.test.ts: provider cost preferred, token-total fallback, missing metering bills zero, and output-only usage tripping an input-blind limit.npm testbudget suite 8/8 pass; full existing budget tests unchanged and green.npm run typecheckclean, eslint/oxlint clean on touched files.This change was prepared with AI assistance under human direction and review.
Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.