Skip to content

fix: name the field or reason in four failed tool results (#2149) - #2150

Merged
hidb4ai merged 4 commits into
oceanbase:masterfrom
guamiao:fix/2149-tool-error-details
Oct 23, 2026
Merged

hidb4ai merged 4 commits into
oceanbase:masterfrom
guamiao:fix/2149-tool-error-details

Conversation

@guamiao

@guamiao guamiao commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Closes #2149.

Rationale for this change

The agent tool recovery evaluation (#2067 / #2146) found four failed PowerContext tool results that did not tell the agent what to change: three Server 422 responses omitted the rejected field, and the Codex plugin reported an authorization denial as a Scope-binding failure. Agents guessed, retried with changed arguments, or told the user the wrong cause.

What changes are included in this PR?

  • Dream evidence selection (create_dream_run): an empty artifacts selection reports loc=["artifacts"] with type evidence_selection_required; more than 20 Artifacts reports artifacts with evidence_limit_exceeded; more than 32 references counting Sources reports sources with evidence_limit_exceeded. Messages state the bounds for the selected operation: revise_tags has a combined limit of 32 references and no separate 20-Artifact limit.
  • Candidate validation (propose_experience): InvalidCandidateError details map the internal artifacts field to the public artifact_refs field with a stable reason (target_not_in_artifact_refs), and name target for family/target failures. Internal-only fields and reasons shared by several public fields keep details: null; no exception text is echoed.
  • Working-note expiry (capture_working_note): a past expiry returns {"field": "expires_at", "reason": "not_in_future"}, and the runtime separates working-note-expiry-timezone from working-note-expiry-past. Replay of an existing note is unchanged.
  • Codex plugin: the PreToolUse hook now classifies 401 as an authentication failure and 403 as an authorization denial, keeps transport and unresolved-binding failures on the existing reason, and catches protocol-level http.client errors so the hook denies instead of crashing.
  • Tests: field-level validation tests for the Dream bounds, mapping tests for candidate errors, HTTP-level assertions for the 422 bodies (including the revise_tags boundary), and direct runtime tests for both expiry reasons.

Status codes and outer error codes are unchanged. No OpenAPI or generated model changes were required (scripts/generate_api.py --check passes).

Are there any user-facing changes?

Error details now carry a field name and a stable reason so an agent can correct the request. No endpoints or request fields change. Known limits:

  • Candidate errors with no unique public field (evidence, basis, audit, family) and the approve-only lineage sources error keep details: null.
  • A naive expires_at is rejected by the transport model as timezone_aware; the field/reason shape applies to the past-expiry path and to direct runtime calls.
  • A pre-existing admission/execution mismatch remains for revise_tags: the runtime appends tag_target.basis_ref to the resolved Artifacts, so a 32-reference selection is only valid end to end when that basis Artifact is already part of the selection. This PR only corrects the admission message; the counting mismatch should be handled separately.

How was this change tested?

  • pytest tests/builtin/dream tests/builtin/review tests/e2e/test_builtin_runtime.py tests/e2e/test_runtime_server.py tests/e2e/test_artifact_dreaming.py tests/e2e/test_candidate_review.py tests/test_api_contract.py tests/test_error_details.py tests/e2e/test_request_error_details.py -q → 194 passed, 82 skipped (before the review follow-up).
  • pytest tests/builtin/dream tests/e2e/test_request_error_details.py -q → 11 passed (after the review follow-up).
  • pytest tests/codex_plugin/test_scope_binding.py -q → 84 passed.
  • ruff check, ruff format --check, and ty check on the changed files pass.
  • python scripts/generate_api.py --check passes.
  • Known unrelated Windows failure: tests/builtin/runtime/test_extraction_diagnostics.py::test_extraction_diagnostics_observes_spawned_worker_failure_and_stopped_supervisor[_worker_crash-...] also fails on a clean upstream/master checkout.

AI usage statement

AI-assisted: Claude Code (Anthropic, Claude Sonnet 4.6) was used to implement the change and write the tests. The resulting diff was reviewed by two independent agent reviewers, and the reviewer P3 finding was addressed in commits 3835fbda and 641469f8.

guamiao and others added 2 commits October 21, 2026 15:23
…ceanbase#2149)

The PreToolUse hook mapped every 401/403 from the Scope binding lookup to the
generic "could not resolve the current Scope binding" reason, so agents told
the user the binding was broken instead of naming the rejected credential.
Classify the denials the same way session_binding.py does, keep transport and
unresolved-binding failures on the existing reason, and catch protocol-level
http.client errors so the hook denies instead of crashing.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
…anbase#2149)

Four failed tool results left agents to guess what to change. Dream evidence
selection now reports the rejected field and both bounds, separating an empty
selection from too many Artifacts and from the combined reference limit.
Candidate validation maps the internal artifacts field to the public
artifact_refs field with a stable reason instead of returning no details.
Working-note expiry names expires_at and distinguishes a missing time zone from
a past expiry. Status codes and error codes are unchanged.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

@hidb4ai hidb4ai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One non-blocking P3 issue in the evidence-limit diagnostic.

@AlexStocks AlexStocks added the 1.4.1 v1.4.1 release tracking label Oct 21, 2026
guamiao and others added 2 commits October 22, 2026 10:27
…ase#2150)

revise_tags has a combined 32-reference limit and no separate 20-Artifact
limit, so its rejection message must not tell the agent to discard valid
Artifact evidence. Keep the operation-specific bounds in the field error and
cover both the rejecting and the accepted boundary.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
…ase#2150)

Exercise the operation-specific rejection and accepted boundary through the
public route, and assert the exact response details instead of a status check
that also passes on a missing Scope. Unify the lower-bound wording with the
combined reference limit.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

@hidb4ai hidb4ai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@hidb4ai
hidb4ai merged commit bb3dcfc into oceanbase:master Oct 23, 2026
52 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

1.4.1 v1.4.1 release tracking

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix: name the field or denial in four failed tool results that agents could not act on

3 participants