diff --git a/reference/configuration/options.md b/reference/configuration/options.md index 0c74b158..3f09ed0c 100644 --- a/reference/configuration/options.md +++ b/reference/configuration/options.md @@ -404,8 +404,9 @@ agent: - `enabled` — Enable the agent component; _Default_: `false` - `provider` — Recorded on the session but **not yet used to route the model call** — only `model` reaches the provider. Set the provider through the [`models`](../models/overview.md#configuration) configuration instead - `model` — Model id override, passed through to the model call; _Default_: the [`models`](../models/overview.md#configuration) generative default -- `maxTurns` — Maximum tool-call iterations in a single run; _Default_: `50` -- `maxCostUsd` — Intended per-run cost ceiling. **Not enforced** — it is a stored setting only, and nothing checks spend against it; _Default_: `5.00` +- `maxTurns` — Maximum tool-call iterations in a single run; a positive integer, validated from v5.4.0. A bare `maxTurns:` keeps the default; _Default_: `50` +- `maxSessionTokens` — Per-session cap on prompt plus completion tokens, summed over every model call in the session that returns; a positive integer. See [Spend caps](#agent-spend-caps); _Default_: none +- `maxCostUsd` — Per-session cap in USD, priced from the [`pricing`](../models/overview.md#pricing) on the `models.generative` entry that served each call; a number of `0` or more, and `0` stops every session before its first call. See [Spend caps](#agent-spend-caps). Before v5.4.0 this setting was stored but never enforced, and defaulted to `5.00`; _Default_: none - `autoApprove` — Run without per-action approval gates; _Default_: `false` - `allowDestructive` — Include the tools marked destructive in the agent's toolset: `write_file`, the inspector's code-evaluation tools, and any operations tool carrying MCP's [`destructiveHint`](../mcp/tool-metadata.md) (`drop_table`, `delete`, `restart`, `set_configuration`, ...). When `false` they are removed entirely rather than gated. That hint comes from a curated set in core which does not cover every damaging operation, so this is not a complete safety boundary on its own — see [Agent operations](../operations-api/operations.md#agent); _Default_: `false` - `user` — Harper user the agent's **operations** tools run as; the filesystem, HTTP, schedule, and inspector tools always run at process privilege regardless. If it cannot be resolved and it is not the default, the agent fails closed and runs with no operations tools; _Default_: `hdb_agent`, which falls back to a `super_user` bootstrap identity @@ -413,7 +414,26 @@ agent: - `httpFetch` — Whether the agent has its `http_fetch` tool, and which hosts it may reach: `true`, `false`, or `{ allow: [...] }`. Read at startup only. See [Restricting `http_fetch`](#restricting-http_fetch); _Default_: `true` - `systemPromptAppend` — Operator text appended to the agent's system prompt -`enabled`, `provider`, `model`, `maxTurns`, `maxCostUsd`, `autoApprove`, `allowDestructive`, and `systemPromptAppend` can also be changed at runtime with [`set_agent_config`](../operations-api/operations.md#set_agent_config), which applies in memory only. `enabled` is the exception worth knowing: it cannot switch the agent on, because with the agent disabled at startup no agent operation is registered at all. +`enabled`, `provider`, `model`, `maxTurns`, `maxSessionTokens`, `maxCostUsd`, `autoApprove`, `allowDestructive`, and `systemPromptAppend` can also be changed at runtime with [`set_agent_config`](../operations-api/operations.md#set_agent_config), which applies in memory only. `enabled` is the exception worth knowing: it cannot switch the agent on, because with the agent disabled at startup no agent operation is registered at all. + +### Agent spend caps + + + +`maxSessionTokens` and `maxCostUsd` bound a whole session, across every run in it. Approving a paused action or sending a follow-up prompt starts a new run, but does not reset a cap the way it resets `maxTurns`. Each assistant message records the usage of the model call that produced it, and [`get_agent_session`](../operations-api/operations.md#get_agent_session) reports the session's totals as `usage`. + +A cap is checked when a run starts, before it executes an approved action, and after every model call: + +- **At or over a cap**, the session makes no further model call and ends `error`, with `lastError` naming the cap, for example `agent.maxSessionTokens=200000 reached: this session used 201450 tokens`. Tool calls the model had just requested are answered with a `budget_exceeded` result rather than run, so the transcript stays valid if you continue the session. A final answer that crosses the cap is still delivered: the run ends `completed`, with `lastError` naming the cap. +- **A cap that cannot be measured stops the session too.** `maxCostUsd` needs every call priced, and `maxSessionTokens` needs every call's token counts. A model with no `pricing`, or a provider that reports no usage, stops the session after its first call rather than letting it run uncapped: it ends `error`, or `completed` if that call was the final answer, with `lastError` saying why either way. The unmeasured call stays in the transcript, so the session's total stays unmeasured and later runs stop before making any call, and raising the cap or adding `pricing` does not clear it: start a new session, or clear the cap. `set_agent_config` with `null` clears it until the next restart; to clear it for good, remove the key from the configuration file. A session created before v5.4.0, whose earlier calls recorded no usage, is in the same state. +- **Pending approvals are closed too.** A run stopped before it executes an approved action answers that action, and any approval still awaiting a decision, with `budget_exceeded`, so nothing is left pending. The action does not run later, even once the cap is raised; the model sees the refusal and can ask again. +- **A capped session stays capped.** Raise the cap with `set_agent_config` or start a new session. + +A cap is a threshold checked after each call, not a guarantee: the call that reaches it can overshoot it. Only calls that return are counted. A call that fails, for example on a timeout or a provider error and whether or not another model then serves it, is not, even if the provider billed it. + +A cap, or a `maxTurns`, that is not a usable number fails the agent's start: Harper logs the error and registers no agent operation. Through `set_agent_config` the same value is rejected with a 400. + +If you upgrade with `maxCostUsd` already set in your configuration, it starts being enforced, and needs `pricing` on the generative entry the agent uses. ### Restricting `http_fetch` diff --git a/reference/models/api.md b/reference/models/api.md index 29ffbf1d..8f9e51a7 100644 --- a/reference/models/api.md +++ b/reference/models/api.md @@ -68,13 +68,14 @@ Additional options apply only when `toolMode: 'auto'`; they are documented in [T ### GenerateResult -| Field | Type | Description | -| -------------- | -------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- | -| `content` | `string` | The generated text | -| `finishReason` | `'stop'` \| `'length'` \| `'tool_calls'` \| `'content_filter'` | Why generation stopped, normalized across backends | -| `toolCalls` | `ToolCall[]` | Tool calls the model requested, when `finishReason` is `'tool_calls'` (each `{ id, name, arguments }`, with `arguments` parsed to an object) | -| `usage` | `TokenUsage` | Token usage reported by the backend (`promptTokens`, `completionTokens`, …), when available | -| `trace` | `ToolTraceEntry[]` | Per-tool-invocation trace; only populated by the `toolMode: 'auto'` loop — see [Tool Calling](./tool-calling) | +| Field | Type | Description | +| -------------- | -------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `content` | `string` | The generated text | +| `finishReason` | `'stop'` \| `'length'` \| `'tool_calls'` \| `'content_filter'` | Why generation stopped, normalized across backends | +| `toolCalls` | `ToolCall[]` | Tool calls the model requested, when `finishReason` is `'tool_calls'` (each `{ id, name, arguments }`, with `arguments` parsed to an object) | +| `usage` | `TokenUsage` | Token usage reported by the backend (`promptTokens`, `completionTokens`, …), when available | +| `costUsd` | `number` | From v5.4.0, the call's cost from the serving entry's [pricing](./overview#pricing); absent when that entry is unpriced or usage is incomplete. With `toolMode: 'auto'`, the final round's only, like `usage` | +| `trace` | `ToolTraceEntry[]` | Per-tool-invocation trace; only populated by the `toolMode: 'auto'` loop — see [Tool Calling](./tool-calling) | ## generateStream() @@ -82,7 +83,7 @@ Additional options apply only when `toolMode: 'auto'`; they are documented in [T models.generateStream(input: GenerateInput, options?: GenerateOpts): AsyncIterable ``` -Identical to `generate()` but yields the completion incrementally: +Identical to `generate()` but yields the completion incrementally. Streamed calls are not [priced](./overview#pricing), so no chunk carries `costUsd`: ```javascript let text = ''; diff --git a/reference/models/backends.md b/reference/models/backends.md index 19b34e65..be4b7188 100644 --- a/reference/models/backends.md +++ b/reference/models/backends.md @@ -26,6 +26,7 @@ All backends support these common fields: | `model` | Provider-side model identifier (e.g. `gpt-4o`) used when a call does not pass its own `model` option | | `requestTimeoutMs` | Per-request timeout in milliseconds; composed with any caller-provided `AbortSignal` | | `revision` | From v5.3.1, a non-secret label for the deployment behind the entry, for [calibration](./calibration#when-the-deployment-changes-but-the-configuration-does-not); change it when the same settings reach a different model | +| `pricing` | From v5.4.0, `generative` entries only: `{ inputPerMtok, outputPerMtok }`, what you pay in USD per million prompt and completion tokens. See [Pricing](./overview#pricing) | ## Ollama diff --git a/reference/models/overview.md b/reference/models/overview.md index 0e0ae29b..e389c364 100644 --- a/reference/models/overview.md +++ b/reference/models/overview.md @@ -145,6 +145,30 @@ The logical name `default` is used when application code does not pass an explic See [Backends](./backends) for the full set of configuration fields supported by each backend. +### Pricing + + + +A `generative` entry can declare what you pay for its model, so Harper can put a dollar figure on each call. Harper ships no prices of its own: the rates are yours to keep current. + +```yaml +models: + generative: + default: + backend: openai + apiKey: ${OPENAI_API_KEY} + model: gpt-4o + pricing: + inputPerMtok: 2.5 # USD per million prompt tokens + outputPerMtok: 10 # USD per million completion tokens +``` + +Each `generate()` call is priced at the rates of the entry that served it, and the result carries `costUsd`. With `toolMode: 'auto'`, the result's `costUsd` covers the final round only, like its `usage`. A call that fell back to another entry in a [fallback group](./routing#fallback-groups) is priced at that entry's rates, not the requested one's. A call has no `costUsd` when the entry that served it declares no `pricing`, or when the provider did not report both a prompt and a completion token count. + +The two rates are the only cost inputs Harper uses. Discounts for cached prompt tokens, long-context surcharges and provisioned capacity are not, so set rates that match what an average call costs you, or rates that overestimate it if the figure backs a cap. Streaming calls (`generateStream()`) are not priced. + +The price is read by [`maxCostUsd`](./tool-calling#options) on `toolMode: 'auto'` calls and by the [built-in agent's `maxCostUsd`](../configuration/options#agent-spend-caps). `pricing` is accepted only on `generative` entries; on an `embedding` or `decision` entry it fails configuration validation. A price change applies on config reload, except on an entry whose `backend` is a module rather than a built-in, which takes it on restart like any other change to that entry. It does not affect [calibration](./calibration). + ### Credentials String values in model entries support environment-variable indirection with `${VAR_NAME}` syntax, resolved at startup. Use this for API keys rather than placing the literal key in the configuration file — Harper logs a warning at startup when a credential field contains a literal value. If the referenced environment variable is unset, the placeholder is left as-is; for credential fields the backend rejects the unresolved placeholder at startup, while other fields (such as `host` or `model`) carry the literal placeholder into requests — surfacing as per-request failures rather than a startup error. Indirection applies to string-typed fields only; numeric fields such as `requestTimeoutMs` must be literal values. diff --git a/reference/models/tool-calling.md b/reference/models/tool-calling.md index 0c0f1174..b7d3a3c1 100644 --- a/reference/models/tool-calling.md +++ b/reference/models/tool-calling.md @@ -78,20 +78,21 @@ A handler receives the parsed arguments and a context object `{ signal, accounti All options below apply only with `toolMode: 'auto'`. -| Option | Type | Default | Description | -| -------------------- | ----------------------------- | ------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `toolHandlers` | `Record` | — | Dispatch table keyed by tool name. A tool declared in `tools` with no handler here is a configuration error (400) | -| `maxToolIterations` | `number` | `10` | Hard cap on model → tools → model rounds | -| `maxToolTokens` | `number` | — | Cumulative prompt+completion token cap across rounds. Best-effort: requires the backend to report usage; if it does not, Harper warns once and `maxToolIterations` remains the bound. Not supported on `generateStream()` (throws) | -| `toolParallelism` | `'parallel'` \| `'serial'` | `'parallel'` | When one round requests multiple tool calls, run handlers concurrently or in order | -| `toolResultMaxBytes` | `number` | `65536` | Per-result byte cap (JSON-stringified). Larger results are truncated with a marker; the model sees the truncated form, the trace records the original size | -| `toolErrorMode` | `'recover'` \| `'abort'` | `'recover'` | `'recover'` feeds a handler error back to the model as the tool result so it can react; `'abort'` stops the loop and throws with the trace attached | -| `includeToolTrace` | `boolean` | `false` | Populate `result.trace` with one entry per tool invocation (iteration, name, arguments, result size, duration, error) | -| `conversation` | `ConversationAppender` | — | Optional persistence hook — see below | +| Option | Type | Default | Description | +| -------------------- | ----------------------------- | ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `toolHandlers` | `Record` | — | Dispatch table keyed by tool name. A tool declared in `tools` with no handler here is a configuration error (400) | +| `maxToolIterations` | `number` | `10` | Hard cap on model → tools → model rounds | +| `maxToolTokens` | `number` | — | Cumulative prompt+completion token cap across rounds. Best-effort: requires the backend to report usage; if it does not, Harper warns once and `maxToolIterations` remains the bound. Not supported on `generateStream()` (throws) | +| `maxCostUsd` | `number` | — | From v5.4.0, cumulative USD cap across rounds, from the [pricing](./overview#pricing) of the entry that served each round. If a round is unpriced, Harper warns once and `maxToolIterations` remains the bound; a cap of `0` still stops after the first round. This is the opposite of the built-in agent's `maxCostUsd`, which stops rather than run unpriced. Not supported on `generateStream()` (throws) | +| `toolParallelism` | `'parallel'` \| `'serial'` | `'parallel'` | When one round requests multiple tool calls, run handlers concurrently or in order | +| `toolResultMaxBytes` | `number` | `65536` | Per-result byte cap (JSON-stringified). Larger results are truncated with a marker; the model sees the truncated form, the trace records the original size | +| `toolErrorMode` | `'recover'` \| `'abort'` | `'recover'` | `'recover'` feeds a handler error back to the model as the tool result so it can react; `'abort'` stops the loop and throws with the trace attached | +| `includeToolTrace` | `boolean` | `false` | Populate `result.trace` with one entry per tool invocation (iteration, name, arguments, result size, duration, error) | +| `conversation` | `ConversationAppender` | — | Optional persistence hook — see below | ### Budgets and errors -When `maxToolIterations` or `maxToolTokens` is exhausted, the loop throws a budget-exceeded error (HTTP status 429) carrying a `partialTrace` of everything that ran — the trace is attached on error paths regardless of `includeToolTrace`. With `toolErrorMode: 'abort'`, a handler failure throws an error carrying the same trace. +When `maxToolIterations`, `maxToolTokens` or `maxCostUsd` is exhausted, the loop throws a budget-exceeded error (HTTP status 429) carrying a `partialTrace` of everything that ran — the trace is attached on error paths regardless of `includeToolTrace`. With `toolErrorMode: 'abort'`, a handler failure throws an error carrying the same trace. If the model requests a tool name that was never declared in `tools` (a hallucinated tool), the call is treated as a tool error and routed by `toolErrorMode` — with `'recover'`, the model is told the tool is unknown and can correct itself. @@ -101,4 +102,4 @@ The `conversation` option accepts any object with an `append(turn)` method retur ### Reserved options -`toolArgValidation` (`'strict'` / `'lenient'` JSON Schema validation of tool arguments), `maxCostUsd`, and `conversationId` exist on the type surface but are not functional in 5.1 — the validation modes and streaming token budgets throw a `501` error, and the cost cap has no rate card behind it yet. Don't rely on them. +`toolArgValidation` (`'strict'` / `'lenient'` JSON Schema validation of tool arguments) and `conversationId` exist on the type surface but are not functional: the validation modes and streaming token and cost budgets throw a `501` error. Don't rely on them. Before v5.4.0, `maxCostUsd` had no prices behind it and never tripped. diff --git a/reference/operations-api/operations.md b/reference/operations-api/operations.md index 23e1e330..4a29af86 100644 --- a/reference/operations-api/operations.md +++ b/reference/operations-api/operations.md @@ -1742,9 +1742,9 @@ A session's `status` is one of: | `awaiting_approval` | Paused on one or more destructive tool calls; see `pendingApprovals` | | `completed` | The run ended without throwing — a final answer, or the `maxTurns` ceiling; check `lastError` | | `aborted` | Cancelled by an operator via `cancel_agent_run` | -| `error` | The run failed; `lastError` carries the message | +| `error` | The run failed or a spend cap stopped it; `lastError` carries the message | -`completed` also covers hitting the `agent.maxTurns` ceiling — in that case `lastError` reads `Reached maxTurns= without a final answer.`, so check it before treating a completed session as finished. +`completed` also covers hitting the `agent.maxTurns` ceiling — in that case `lastError` reads `Reached maxTurns= without a final answer.`, so check it before treating a completed session as finished. From v5.4.0 a final answer that reached a [spend cap](../configuration/options.md#agent-spend-caps) also ends `completed` with `lastError` set, naming the cap. ### `agent_prompt` @@ -1768,12 +1768,24 @@ Response: { "session_id": "3f7c...", "status": "running" } ``` -A session that is `running` or `awaiting_approval` rejects a new prompt with a 409 — resolve the pending approval or cancel the run first. +A session that is `running` or `awaiting_approval` rejects a new prompt with a 409 — resolve the pending approval or cancel the run first. From v5.4.0 a session also answers 409 while a cancelled run is still finishing its current model call or tool. The session already reads `aborted` then, so retry with backoff; the wait lasts as long as that call, at most the model's `requestTimeoutMs` or the tool's own run time. A prompt accepted while a [`cancel_agent_run`](#cancel_agent_run) for the same session arrives is recorded but not run, and answers 409 too. ### `get_agent_session` Returns the full session record: `status`, `user` (the Operations API caller who created the session, falling back to `agent.user`), the `messages` transcript (user, assistant, and tool messages, including tool calls and their observations), `pendingApprovals`, `model`, `provider`, `createdAt`/`updatedAt`, and `lastError`. This is the polling endpoint for a run in flight. +From v5.4.0 each assistant message carries the `usage` its model call reported (`promptTokens`, `completionTokens`, and `costUsd` when the model is [priced](../models/overview.md#pricing)), and the response adds the session's totals: + +```json +{ + "session_id": "3f7c...", + "status": "completed", + "usage": { "calls": 3, "promptTokens": 41230, "completionTokens": 2210, "costUsd": 0.157 } +} +``` + +`calls` counts the model calls that returned an answer; a failed call is not counted. A total is left out once any call in the session did not report it, such as an unpriced model for `costUsd`, so a missing total means unmeasured, not zero. The [spend caps](../configuration/options.md#agent-spend-caps) read the same totals. + ```json { "operation": "get_agent_session", "session_id": "3f7c..." } ``` @@ -1782,7 +1794,7 @@ Unknown `session_id` returns a 404. ### `list_agent_sessions` -Lists agent sessions. +Lists agent sessions, each with the same `usage` totals as `get_agent_session`. | Parameter | Type | Description | | --------- | ------- | ---------------------------------------- | @@ -1827,6 +1839,8 @@ Whether a tool is treated as destructive at all is governed by `agent.allowDestr Cancels a session's run, clears any followups it scheduled, and marks the session `aborted`. Works on a paused (`awaiting_approval`) or `idle` session as well as an actively running one. +From v5.4.0 the cancel also answers every pending approval, decided or not, with a `cancelled_by_operator` result, in the same write that marks the session `aborted`. An approved action that has not started by then never runs, neither in the cancelled run nor when you next prompt the session, and nothing is left awaiting a decision. One already running finishes, and its result is recorded. A cancel between a turn's tool calls answers the rest of that turn `cancelled_by_operator`. The session stays `aborted`: a run caught mid-step finishes that step without marking the session `completed`, `error` or `awaiting_approval`, a tool call it then asks to have approved is answered `cancelled_by_operator` instead, and it makes no further model call. + ```json { "operation": "cancel_agent_run", "session_id": "3f7c..." } ``` @@ -1843,7 +1857,7 @@ One gap is worth knowing: changing `allowDestructive` with [`set_agent_config`]( ### `set_agent_config` -Updates agent settings and returns the resulting configuration. Accepts any of `enabled`, `provider`, `model`, `maxTurns`, `maxCostUsd`, `autoApprove`, `allowDestructive`, and `systemPromptAppend`; keys not supplied are left unchanged. Each field is described under [`agent`](../configuration/options.md#agent). A request that includes `httpFetch` is rejected with a 400 and nothing in it is applied: the [`http_fetch` policy](../configuration/options.md#restricting-http_fetch) is read at startup only. +Updates agent settings and returns the resulting configuration. Accepts any of `enabled`, `provider`, `model`, `maxTurns`, `maxSessionTokens`, `maxCostUsd`, `autoApprove`, `allowDestructive`, and `systemPromptAppend`; keys not supplied are left unchanged. Set `maxSessionTokens` or `maxCostUsd` to `null` to remove that cap. `maxTurns` must be a positive integer. A value that is not usable is rejected with a 400, and nothing in the request is applied. Each field is described under [`agent`](../configuration/options.md#agent). A request that includes `httpFetch` is rejected with a 400 and nothing in it is applied: the [`http_fetch` policy](../configuration/options.md#restricting-http_fetch) is read at startup only. ```json { "operation": "set_agent_config", "autoApprove": false, "maxTurns": 20 } @@ -1852,7 +1866,7 @@ Updates agent settings and returns the resulting configuration. Accepts any of ` Three limits are worth knowing: - **The change is in-memory and not persisted.** It applies for the life of the process and is lost on restart; edit `harper-config.yaml` for a durable change. -- **A run already in flight keeps the settings it started with** — its toolset, `autoApprove`, `model`, and `systemPromptAppend` are all captured at start. Changes take effect on the next run. To stop a run immediately, use `cancel_agent_run`. +- **A run already in flight keeps the settings it started with** — its toolset, `autoApprove`, `model`, `maxTurns`, `systemPromptAppend` and spend caps are all captured at start. Changes take effect on the next run. To stop a run immediately, use `cancel_agent_run`. - **`enabled` is not a kill switch.** It cannot turn the agent on — if it was off at startup, this operation does not exist. Setting it to `false` only makes subsequent `agent_prompt` calls return 409; a run already in flight continues, and `approve_agent_action` still resumes a paused one. Use `cancel_agent_run` to stop a run. ### MCP access diff --git a/release-notes/v5-lincoln/5.4.md b/release-notes/v5-lincoln/5.4.md index 32120367..4ed87010 100644 --- a/release-notes/v5-lincoln/5.4.md +++ b/release-notes/v5-lincoln/5.4.md @@ -23,3 +23,17 @@ A durable MQTT session now resumes through the same checked replay. When a recon ### Certifying a Release in a Canary Worker A `deploy_component` with `restart: true` or `restart: "rolling"` now checks its release before rolling it out. The first worker started on the release, the canary, is held out of traffic until it reports whether the deployed component loaded, and the rollout goes on only if it did. Each later replacement is checked the same way. When the node that received the deploy rejects the release, the deploy fails with `400`. The release it replaced is made live again, or, with nothing to go back to, the component fails closed on that node. A new `certification` field in the response reports the outcome, and says when a restarting deploy went out unchecked (`uncertified` or `unavailable`, as for a `package: file:` deploy). With `"rolling"`, each other node then certifies the release with a canary of its own, and a node that rejects it fails the deploy's job, not the deploy: poll its `restartJobId` with `get_job` for the outcome. A node running a version before 5.4.0 activates without a canary. A deploy that restarts nothing is not test-loaded. If its release throws when it loads, `get_status` reports the failure. See [Certifying a release in a canary worker](/reference/v5/operations-api/operations#certifying-a-release-in-a-canary-worker). + +## Models and the Built-in Agent + +### Model Pricing and Per-Call Cost + +A `models.generative` entry can now declare `pricing: { inputPerMtok, outputPerMtok }`, what you pay in USD per million prompt and completion tokens. Harper ships no prices of its own. Each `generate()` call is priced at the rates of the entry that served it, a fallback included, and the result carries `costUsd` when that entry is priced and the provider reported both token counts. With `toolMode: 'auto'` it covers the final round, and streaming calls are not priced. This makes `maxCostUsd` work on `toolMode: 'auto'` calls; before, it had no prices behind it and never tripped. See [Pricing](/reference/v5/models/overview#pricing). + +### Spend Caps for the Built-in Agent + +The agent records the usage of each model call on its session, and `get_agent_session` and `list_agent_sessions` report a session's totals as `usage`. Two caps bound a whole session, across every run in it: the new `agent.maxSessionTokens`, and `agent.maxCostUsd`, which is now enforced from model pricing and no longer defaults to `5.00`. A session at a cap makes no further model call: it ends `error`, or `completed` with `lastError` naming the cap when the call that reached it was the final answer. A cap that cannot be measured stops the session rather than letting it run uncapped. Caps count calls that return: a call that fails, even one the provider billed, is not counted. If your configuration already sets `agent.maxCostUsd`, declare `pricing` on the generative entry the agent uses, and start new sessions: a session created before 5.4.0 recorded no usage, so it cannot be continued under a cap. See [Agent spend caps](/reference/v5/configuration/options#agent-spend-caps). + +### Agent Cancellation and Limits + +`cancel_agent_run` now answers every pending approval with a `cancelled_by_operator` result in the same write that marks the session `aborted`, so an approved action that has not started never runs, then or on the session's next prompt, and the session stays `aborted` even when a run was mid-step. `agent.maxTurns` is now validated as a positive integer at startup and by `set_agent_config`, where a value such as `null` used to make every later run end without a model call. See [`cancel_agent_run`](/reference/v5/operations-api/operations#cancel_agent_run).