Skip to content

fix(budget): bill metered LLM usage instead of a flat input-only estimate - #662

Closed
Mr-Neutr0n wants to merge 1 commit into
yc-software:mainfrom
Mr-Neutr0n:fix/budget-metered-usage
Closed

fix(budget): bill metered LLM usage instead of a flat input-only estimate#662
Mr-Neutr0n wants to merge 1 commit into
yc-software:mainfrom
Mr-Neutr0n:fix/budget-metered-usage

Conversation

@Mr-Neutr0n

@Mr-Neutr0n Mr-Neutr0n commented Aug 23, 2026

Copy link
Copy Markdown

Fixes #586

Summary

Turn-level budget accounting now uses the metered usage each harness already reports per LLM request instead of a single flat, input-only estimate.

Root cause

recordModelCall fires once per turn with an estimated input-token count only, and the budget recorded estimateCostUsd(rec.inputTokens) from it: every model priced at DEFAULT_AGENT_INPUT_USD_PER_MTOK (5), no output or cache tokens. The measured numbers existed on a second callback the budget never saw: recordLlmRequest receives usage (input, output, cacheRead, cacheWrite, provider-computed costUsd) for every step.

What changed

  • costFromUsage() in src/ratelimit/budget.ts: prefers the provider-reported costUsd, falls back to (input+output+cacheRead+cacheWrite) at the default rate when tokens are reported without cost, and returns 0 when nothing is reportable.
  • The main turn path records budget from that metered usage inside the existing recordLlmRequest handler; the once-per-turn estimate is removed there so steps are not double-counted. Detect, compaction, and security-screen paths keep their existing estimates (they have no metered counterpart).

Verification

  • 4 new cases in test/budget.test.ts: provider cost preferred, token-total fallback, missing metering bills zero, and output-only usage tripping an input-blind limit.
  • npm test budget suite 8/8 pass; full existing budget tests unchanged and green.
  • npm run typecheck clean, eslint/oxlint clean on touched files.

This change was prepared with AI assistance under human direction and review.


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

…mate

The main turn path recorded one estimateCostUsd(inputTokens) per turn,
pricing every model at DEFAULT_AGENT_INPUT_USD_PER_MTOK and ignoring
output/cache tokens entirely, while the per-step metered usage that each
harness already reports through recordLlmRequest (input, output,
cacheRead, cacheWrite, provider-computed costUsd) never reached the
budget tracker.

Budget accounting for turns now comes from that metered usage via
costFromUsage(): it prefers the provider-reported costUsd and falls back
to a total-token estimate at the default rate when a step reports tokens
but no cost. Steps with no reportable usage bill zero rather than a
guessed figure.

Fixes #586

This change was prepared with AI assistance under human direction and review.
@time-attack

Copy link
Copy Markdown
Collaborator

Thanks for this, @Mr-Neutr0n — and sorry for the slow response.

We're closing this PR. Per CONTRIBUTING.md, since coding agents write most of the underlying code now, we'd rather receive feature ideas as a short human-written note in adrs/ — informal is great, just the idea in your own words — and bugs as a plain issue. If we're aligned we're happy to burn our own tokens on the implementation, and we'll credit you as co-author.

We don't want your idea to get lost, so we've captured it in #748 to triage on our side. Please feel free to open an adrs/ note if you feel strongly about the approach. Appreciate you engaging with the repo!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Budget enforcement prices every model at $5/MTok input-only, ignoring the per-call usage it already records

2 participants