Cap the agent's spend per session by tokens or priced cost, record its model usage, and make a cancel final

[DavidCockerill]

⊙ Problem

The built-in agent's only spend control, agent.maxCostUsd, was stored and never read, and a session recorded none of the token usage its model calls already report. One measured investigation used about 800K tokens, visible only by querying hdb_analytics; maxTurns was the only brake, and it resets on every approval resume.

Closes #3040 (agent: carry per-call token usage onto the session and enforce a budget).

❓ Your call: requirement. As specified in the issue, plus cancellation work it pulled in. Budget stops and cancels both have to leave every tool call answered and never run an approved action twice, and review rounds found the existing cancel path could be overwritten, undone or lose a prompt; those fixes are folded in here rather than split out. Milestoned v5.4 (main), so no backport.

💡 Solution

• Usage is recorded per call and folded per session. Each assistant message carries the usage and costUsd its model call reported, written with the message. get_agent_session and list_agent_sessions add the session's totals as usage, folded from the transcript by a field-by-name copy of the row, so a total cannot drift from the calls. A total is omitted once any call did not report it.
• Operators declare what they pay. A models.generative entry may carry pricing: { inputPerMtok, outputPerMtok }. Models.generate prices each call from the entry that served it (a fallback at its own rate) and returns costUsd. Harper ships no prices. The toolMode: 'auto' loop now reads that cost instead of a stub that returned 0.
• Two per-session caps. agent.maxSessionTokens (new) and agent.maxCostUsd (now enforced, default 5 removed) are checked when a run starts and after every model call. A session at a cap makes no further call and ends error with the cap in lastError; its tool calls are answered budget_exceeded, not run. A cap that cannot be measured stops the session rather than running uncapped.
• A cancel is final. cancel_agent_run answers every pending approval in the write that marks the session aborted. Nothing the loop writes afterwards replaces aborted, a cancel between a turn's tool calls answers the rest of the turn, and a cancel that arrives while agent_prompt or approve_agent_action is still writing fences out the run it would start. agent_prompt answers 409 while a cancelled run is still finishing or when a cancel fenced out its run.
• Limits are validated. maxSessionTokens, maxCostUsd and maxTurns are checked at startup (a malformed value stops the agent from starting) and by set_agent_config (400). maxTurns: null used to make every later run end without a model call.

⚠️ Look hardest: the approval and cancel ordering. An approval is claimed under the session lock before its action runs; every writer that answers approvals decides which are open on the row it read under the lock; and every cancel bumps a per-session count that runs admitted earlier are checked against. That is what keeps one response per tool call and a cancel final when a cancel, a budget stop and a running action race.

⚖️ Alternatives

• Planning review adopted a better alternative. The plan stored a running usage total on the session row; the planning reviewer (Codex) proposed per-call usage on each assistant message, folded on read. Adopted: it cannot drift, needs no partial-report state, and a session from before this change reads as unmeasured rather than free.
• Agent caps live in the agent loop, not in toolMode: 'auto'. The auto loop cannot pause for approval, and its budget is per invocation, so it would reset on every approval resume.
• Pricing is resolved in the facade, keyed by backend instance. Pricing by the requested model name would bill a fallback at the primary's rate; a Harper-maintained rate card would be wrong for negotiated, regional and self-hosted pricing.
• A prompt racing a cancelled run is refused, not queued. Queuing it behind the cancelled run was tried: the cancelled run's in-flight tool result then lands after the new prompt, which a provider rejects.

❓ Your call: fail-closed vs fail-open. The agent stops a session whose cap cannot be measured; the auto loop keeps its existing contract of warning once and continuing. Making the auto loop fail closed is a behavior change for application callers, so it is left out. Easy to flip either way.

❓ Your call: naming. The cumulative cap is agent.maxSessionTokens because End agent runs cut off by the output-token limit as errors, and add agent.maxTokens (#3064) uses agent.maxTokens for the per-request output limit, as the models API does. Whichever of the two PRs lands second must keep usage on any assistant turn it rejects, or the next capped run reads the session as unmeasured.

❓ Your call: at-most-once approvals. A crash between claiming an approval and writing its result leaves that call unanswered, where main would run the approved action again. Chosen because these are operator-approved, often destructive, actions.

❓ Your call: what a cap does not bound. It is a stop-after-call threshold, so the call that reaches it can overshoot. A failed call is never counted, even one the provider billed (failure records carry no usage). An unpriced model is detected only after its first call. Caps are per session; a new session starts from zero, so total spend across sessions is unbounded.

❓ Your call: a bare maxTurns: in the config file keeps the default, while set_agent_config refuses null. Reviewers flagged the asymmetry; the config-file case is a YAML artifact rather than an instruction.

🔧 Changes

Product and architecture tour

What a session reports it used

Can an operator see what a run cost, and trust the number?

Before	After
get_agent_session returns the transcript and status; usage exists only as per-minute rows in hdb_analytics.	Each assistant message carries its call's usage and costUsd, and the response adds usage: { calls, promptTokens, completionTokens, costUsd } folded from those messages.

A session total is present only while every call in the session reported it. An absent total with calls > 0 means unmeasured, never zero.

Example: an unpriced call in a priced session

A session makes two calls on a priced entry, then one on a fallback entry with no pricing.

Outcome: usage keeps calls: 3 and both token totals, but drops costUsd; a maxCostUsd cap then stops the session instead of treating the third call as free.

How a call is priced

Which rate does a call pay when routing falls back to another model?

Pricing follows the backend that answered

Bootstrap attaches each generative entry's pricing to the backend instance it builds, and Models.generate prices the call from the instance that actually served it, after the fallback loop has finished.

flowchart LR
    Config[models.generative entry] --> Boot[bootstrap publishEntry]
    Boot --> Instance[backend instance plus pricing]
    Call[generate call] --> Route[primary then fallbacks]
    Route --> Served[backend that answered]
    Served --> Price[costUsd from its pricing]
    Instance -.-> Served

Loading

A call is priced only from both token counts as non-negative integers; a missing, negative or fractional count leaves it unpriced rather than under-priced. The arithmetic runs after the candidate try/catch, so an accounting fault cannot trigger a second, billed fallback call.

pricing is accepted only on generative entries, built-in or module-backed; an embedding or decision entry with it fails validation. It is left out of the calibration source fingerprint, because a price does not change what a model returns. A price edit applies on reload for built-in entries and on restart for module-backed ones, like any other change to them.

When a cap stops a session

What happens to a run, and to its transcript, when it reaches a cap?

One run under a cap

1. Run start — the persisted usage is folded; a session already at a cap, or unmeasured under one, answers every open approval without running it (budget_exceeded, or denied_by_operator for a denial) and ends error before any call.
2. Model call — the assistant message is built with the call's usage and cost.
3. After the call — the fold includes the new message; under the cap the run continues as before.
4. Stop on a tool-call turn — the message and one budget_exceeded answer per tool call are written in one write; the session ends error, with the cap in lastError.
5. Stop on a final answer — the answer is delivered and the run ends completed, still naming the cap in lastError.

Caps are per session, read from the transcript, so an approval resume or a follow-up prompt cannot reset them. A capped session stays capped until the cap is raised or cleared, or a new session starts.

Making a cancel final

Can anything that races a cancel undo it, run an approved action, or leave a tool call unanswered?

A cancel against a run that is still working

The cancel aborts the live run and, in one locked write, marks the session aborted and answers every open approval; every later write from the run keeps that status.

sequenceDiagram
    participant Op as cancel_agent_run
    participant Run as running loop
    participant Row as session row
    Op->>Op: fenceRuns and abort controller
    Op->>Row: cancelSession aborted plus answers
    Run->>Row: claimApproval refused if closed
    Run->>Row: endRun keeps aborted
    Run-->>Run: no further model call

Loading

An approval is claimed under the session lock before its action runs, so a cancel or a budget stop answers only approvals whose action has not started, and no tool call gets two responses.
Every status the loop writes after running goes through endRun, which never replaces aborted.
A run admitted before a cancel starts nothing, so a cancel that lands while agent_prompt or approve_agent_action is still writing is not undone.

agent_prompt answers 409 while a cancelled run is still finishing a model call or a handler, and when a cancel fenced out the run it would start; in the second case the prompt is recorded but not run. cancelSession decides under the lock and leaves a session that a run already finished as it is. The admission count lives in process memory beside the in-flight runs, so it is never pruned: deleting a session's entry would re-admit a request that took its token before the cancel.

• summarizeSessionUsage — the fold and its unmeasured rule.
• budgetViolation — exceeded and unmeasurable caps.
• finalizeGenerateResult — framework-owned usage and cost.
• stopIfOverBudget — the run-start check.
• consumeResolvedApprovals — claim, then recheck the cancel, then run.
• endRun — the write every later status goes through.
• cancelSession — the cancel's single locked write.
• claimApproval — at-most-once execution.

Files

• agent/budget.ts (new): the session fold, cap checks, and validation of the caps and maxTurns.
• agent/loop.ts: run admission and fencing; no running write for a cancelled run; per-call usage and the post-call check; terminal writes through endRun; the run-start stop; a mid-turn cancel; claim before running; and isRunInFlight.
• agent/session.ts: endRun, cancelSession, closeOpenApprovals, addPendingApproval on an aborted session and claimApproval, which replaces markApprovalConsumed.
• agent/operations.ts: session reads with usage; agent_prompt's 409s; cancel_agent_run; approve_agent_action's admission token; and set_agent_config validation.
• agent/agent.ts: startup validation, the admission check in startRun, caps passed to each run, the cancel fence, and the removed $5 default and unenforced warning.
• agent/types.ts: per-call usage, AgentSessionUsage and the optional caps.
• config-root.schema.json, utility/hdbTerms.ts: maxSessionTokens and maxCostUsd in place of the "NOT YET ENFORCED" text, and AGENT_MAXSESSIONTOKENS.
• resources/models/Models.ts: a completed result with no output as a failed attempt, pricing the served call, finalizeGenerateResult and callCostUsd.
• resources/models/backendRegistry.ts, bootstrap.ts, types.ts: the per-instance pricing map, published at bootstrap from the entry's pricing, ModelPricing and GenerateResult.costUsd.
• resources/models/agentLoop.ts: the unmeasurable warn-once, the cost tally, both token counts required, and the deleted stub.
• resources/models/backendHelpers.ts: one isTokenCount for backend parsing, the facade and the auto loop.
• resources/models/decisionStore.ts: pricing left out of the calibration fingerprint.
• validation/configValidator.ts: the pricing schema, also on module-backed generative entries.
• agent/DESIGN.md, resources/models/DESIGN.md, DESIGN.md: the spend-cap and cancel invariants, the pricing invariant and their index entries.

✅ Verification

End-to-end route: a new integration test drives the built server and the operations API against a fake OpenAI-compatible model with priced and unpriced entries: usage matches what the model reported, both caps stop the session with every tool call answered, an unpriced model is named, and a malformed cap is a 400. It passes on RocksDB and on LMDB (HARPER_STORAGE_ENGINE=lmdb).

Unit coverage, each block mutation-checked (reverting the guard it covers makes it fail):

• budget.test.js (new): the fold, cap checks, and cap and maxTurns validation.
• loop.test.js budgets and cancels: stops on both turn kinds, unmeasured caps, approvals closed on a stop, every loop write keeping aborted, mid-turn cancel, claim races, and the admission fence.
• operations.test.js: set_agent_config validation, session reads through accessor-only rows, cancel_agent_run closing approvals, and agent_prompt's 409s and tokens.
• agent.test.js: a malformed limit stops an enabled agent before it registers anything.
• pricing.test.js (new): priced, unpriced, partial and smuggled usage; fallback attribution; hollow results; bootstrap and reload.
• agentLoop.test.js: cost caps from real pricing, and the warn-once for unpriced, empty, half and negative reports.
• sourceIdentity.test.js and configValidator.test.js: pricing outside the fingerprint, and where pricing is accepted.

Full gates on the final head (68f8df165), run on macOS. Every failure was rerun on a clean build of the merge base (c2f94e561) on the same machine and fails there identically:

gate	passing	failing
test:unit:main	6887	23: the 19 that fail on base (tmpdir paths, git tag signing, native addons, the dependency-bundle test), plus 4 deploy-activation tests that pass on a quiet rerun
test:unit:resources	4270	0
test:integration:all	2297	25 in 7 files (socket-path length, single HTTP worker), the same 25 on base
LMDB: resources, apiTests, bin	3243, 201, 307	bin: 2 deploy by reference, same on base

test:unit:main skipped applicationSpawn.test.js, which hangs on this machine (test:unit:main hangs/flakes on macOS (#2538)). The LMDB apiTests ran without the script's harper stop, to leave other local instances running.

Companion docs: Document model pricing, the agent's per-session spend caps, and final cancels (HarperFast/documentation#727).

_{🤖 Generated by Claude Code (Anthropic Claude Opus 5.5); posted via @DavidCockerill.}

Related PRs: #562 independent, #1770 independent, #1835 independent, #2155 independent, #2170 overlaps (agent/session.ts write path; endRun, cancelSession and claimApproval use the same Resource-level put), #2436 independent, #2446 independent, #2532 overlaps (set_agent_config input schema should include maxSessionTokens and allow null caps), #2752 independent, #2763 independent, #2775 independent, #2901 independent, #2906 independent, #2939 independent, #2986 independent, #2990 independent, #2994 independent, #3064 overlaps (agent loop, config and set_agent_config; it takes agent.maxTokens for the per-request output limit), #3070 independent, #3078 independent, #3082 overlaps (pricing is keyed by backend instance in backendRegistry.ts; a backend served across worker threads needs its pricing attached to the instance each thread calls), #3084 independent, #3086 independent, #3093 independent, #3110 independent, #3117 overlaps (agent loop, session writes and the auto loop; whichever lands second keeps usage on every assistant turn), #3126 independent, #3137 independent, #3138 independent, #3140 independent, #3174 independent, #3139 independent, #3072 independent, #2930 independent, #3148 independent, #3141 independent, #3128 independent, #3119 independent, #3079 independent, #3124 independent, #3123 independent, #3116 independent, #3113 independent, #3118 independent, #3083 independent, #3107 independent, #3098 independent, #2649 independent, #3100 independent, #3103 independent, #3069 independent, #3085 independent, #3057 independent, #3030 independent, #3035 independent, #3065 independent, #3036 independent, #2825 independent, #3021 independent, #2981 independent
Complexity: complicated

_{Review-Coverage: authored=claude; ran=codex,gemini; adjudicated=domain; blocked=cursor-grok(not-installed); declined=cursor-composer,cursor-kimi,cursor-muse; rounds=9; full=1 @ 68f8df1}

_{Review-Attention: read ~7m @ 68f8df1}

[gemini-code-assist]

Code Review

This pull request implements per-session spend caps (maxSessionTokens and maxCostUsd) for the built-in agent, ensuring that usage is dynamically calculated from the session transcript to prevent drift. It introduces a pricing model for generative model configurations, enabling the framework to compute the cost of each call based on the serving backend's rates. Additionally, it enhances the agent loop and session management with robust cancellation fencing to prevent race conditions between cancelled runs and incoming prompts or approvals. Since no review comments were provided, I have no feedback to provide on the changes.