feat(runtime): implement turn-scoped tool_search activation

[likun666661]

Problem

Maka currently controls provider-visible tools through overlapping mechanisms: economy/full mode, predefined load_tools groups, MAKA_DISABLE_DEFERRED_TOOLS, historical connector aliases, and cross-turn ledger seeding. The model-facing contract is difficult to explain, and low-frequency tool schemas can add substantial context cost before the model needs them.

Discussion #3621 reached agreement on the first implementation slice after working through binding authority, search-space discovery, activation timing, schema visibility, and the tradeoff between automatic activation and a separate confirmation command.

Related tracker: #1382. This issue implements the provider-independent first slice agreed in #3621 rather than the original provider-native-first and cross-turn persistence assumptions in #1382.

Desired outcome

Provide one Maka-owned, provider-independent tool_search contract:

tool_search(query, limit?)
  -> bounded top-k matches
  -> successful results become active immediately
  -> complete schemas become visible on the next provider step
  -> activation is monotonic within the current turn
  -> activation is cleared when the turn completes

The search ranker selects which bounded schemas become visible. The model selects which visible tool to execute. Runtime binding and execution checks remain authoritative.

Agreed semantics

Authority and visibility

• The tools actually bound to the current run are the capability ceiling.
• Search metadata and the initial inventory are derived from those bindings.
• A tool name in the inventory is discoverable metadata, not callable visibility.
• A tool is visible for a provider step only when its complete callable definition is present in that request.
• Search never binds a new executable tool and never escapes boundTools.
• Tool visibility does not replace permission or execution-time validation.

Initial discovery surface

The model receives a lightweight inventory of deferred tools before searching:

group:
- canonical_tool_name
- canonical_tool_name

The inventory contains group names and canonical tool names only. Direct tools already have complete definitions and are not repeated. Dynamic client tools are grouped by their derived source metadata.

Search contract

Initial interface:

tool_search({
  query: string,
  limit?: number,
})

The first index should use canonical name and description metadata derived from current bound deferred tools. Exact names may affect ranking but must not change the operation semantics: every successful search uniformly activates the matches it returns.

The ordinary model-facing result stays thin:

{
  "activated": ["browser_snapshot", "browser_click"]
}

Complete schemas must not be duplicated in the ordinary tool result. They enter the next request through the normal provider tool projection.

Repeated and parallel searches

• Successful searches add their matches to the current turn active set.
• Repeated tools are deduplicated.
• Parallel search results in one provider step are unioned.
• The model may continue searching and expanding the active set.
• There is no turn-wide activation budget or unload operation.
• Each individual search remains bounded by result count and schema bytes.
• The binding ceiling is the final upper bound.

Step boundary

A search result cannot rewrite the request in which the search call was emitted.

For a search completed during provider step n:

step n:
  tool_search executes and updates future active tools

step n + 1:
  matching complete schemas enter the provider request

A parallel hidden-tool call in the same step must still fail the step-start availability guard:

tool_search("browser")
browser_click(...)

The search may affect only the next provider request.

Runtime ownership

The correct owner of mutable activation state is the existing per-send() TurnScope in packages/runtime/src/ai-sdk-backend.ts.

class TurnScope {
  readonly activeTools = new Map<string, MakaTool>()

  // existing per-turn state
}

This matches the current lifecycle:

• openTurnScope() creates one scope for one send();
• every provider step and retry in that send shares the scope;
• overlapping turns on one backend have separate scopes;
• cleanupAfterTurn() drops the scope on completion, error, or cancellation.

The backend-scoped ToolAvailabilityRuntime must remain immutable with respect to turn state. It may own bound catalog/index/projection logic, but it must not own mutable activeTools.

Request projection computes the current provider-visible names from:

direct tools
union TurnScope.activeTools
union context-required orchestration tools
intersect current boundTools

The execution guard must read the immutable step-start active set, not the live map mutated by a search that is settling in the same step.

No additional public ActivationReceipt or VisibleSnapshot domain type is required. The active map is the state, the thin tool result reports what changed, and the actual provider request is the visibility fact.

Direct baseline

Keep the agreed frequent baseline direct when those tools are present in the current binding:

• Bash
• Read
• ArchiveRead
• Write
• Edit
• Glob
• Grep
• WebFetch
• AskUserQuestion
• StopBackgroundTask
• tool_search

Runtime-required tools may still be projected directly when current state requires them. Existing exact hosted-execution profiles and caller-provided boundTools remain hard ceilings.

Implementation work

1. Add TurnScope.activeTools: Map<string, MakaTool>.
2. Replace the current economy/group availability policy with direct and searchable bound-tool derivation.
3. Build a cached search index from canonical bound tool name and description metadata.
4. Add the synthetic direct tool_search tool with a per-turn activation callback.
5. Generate the grouped canonical-name inventory from the effective bound deferred surface.
6. Project direct + active + required tools before every provider request.
7. Preserve a step-start gating snapshot so same-step search and hidden-tool execution cannot race.
8. Union and deduplicate successful repeated and parallel searches.
9. Record search query, ranked names, activated names, schema characters, and subsequent actual calls for evaluation.
10. Remove the old model-facing availability mechanisms and update documentation.

Likely primary files:

• packages/runtime/src/ai-sdk-backend.ts
• packages/runtime/src/tool-availability.ts
• packages/runtime/src/tool-catalog-derive.ts
• packages/runtime-host/src/server/interactive-run-composer.ts
• deferred-tool and execution-composition tests

Migration

Remove from the new model-facing contract:

• economy/full mode;
• MAKA_DISABLE_DEFERRED_TOOLS;
• predefined groups as activation units;
• load_tools;
• load_tool;
• connect_tool_source;
• cross-turn activation replay and ledger seeding.

Historical RuntimeEvents may remain readable when required for old transcript compatibility, but they must never seed a new turn active set.

Provider-native tool search, if added later, must remain an optimization behind this same Maka-owned contract rather than a second semantic path.

Acceptance criteria

1. The initial provider request contains the direct baseline and tool_search, but not deferred complete schemas.
2. The lightweight inventory lists only currently bound deferred canonical names.
3. Search results can contain only tools in the current binding ceiling.
4. The ordinary search result contains activated names without complete schemas.
5. Search completed in step n exposes complete matching schemas in step n + 1, never step n.
6. Repeated and parallel searches accumulate a deduplicated active set for the current turn.
7. A same-step hidden-tool call remains rejected even when a parallel search activates it for the next step.
8. Provider retries preserve the current turn active set without double activation.
9. Completion, failure, and cancellation do not leak active tools into another turn.
10. Exact hosted profiles and explicit boundTools cannot be widened by search.
11. Direct and discovered tools cross the same permission, durable execution, output bounding, and artifact boundaries.
12. New turns do not restore activation from historical load_tools or alias calls.
13. Tests cover direct mode, code mode, graph/swarm required tools, dynamic client tools, retry, parallel calls, and turn cleanup.
14. Telemetry can report exposed-but-unused tool count/schema characters, top-k recall, repeated searches, and time to first successful tool call.

Evaluation

Measure:

• schemas exposed but never called;
• schema characters exposed but unused;
• top-k recall of the tool eventually called;
• revised or repeated searches;
• time to the first successful tool call;
• tasks where the required tool falls outside the per-search activation cap.

If the measurements show material context waste or retrieval misses, evaluate restoring an explicit model confirmation boundary through load_tools. The first slice intentionally starts with uniform search-and-activate semantics.

Explicitly out of scope

• Skill and SkillSearch discovery;
• goal-tool redesign;
• provider prompt-cache semantics;
• provider-native search as a separate contract;
• unload semantics;
• cross-turn discovery persistence;
• changing ToolRuntime execution authority.

References

• Discussion #3621: Simplify Maka tool discovery
• [RFC] Provider-native Tool Search for large tool catalogs #1382
• refactor(runtime-host): unify tool ceiling projection #3620

AI assistance disclosure: Maka helped inspect the current Runtime lifecycle, synthesize the agreement from Discussion #3621, and draft this implementation issue. I reviewed and approved the final scope.

[Astro-Han]

Thanks — this is a great discussion, and I think we are quite close. One thought after tracing the current implementation: perhaps the implementation could be a little simpler:

• TurnScope may only need a Set of activated canonical names, with the bound catalog remaining the schema authority.
• The direct/searchable classification could live in one catalog-owned place.
• The search index could be derived from the immutable bound catalog, without introducing a separate cache abstraction for now.
• The old group/economy/replay state could go away, while load_tools remains only for rendering historical transcripts.

That would leave a fairly small core:

bound catalog → inventory → tool_search → activated-name set → next-step projection → clear at turn end

For v1, I also wonder if per-search limits plus telemetry are enough. We could always introduce a turn-wide budget later if real traces show that it is needed. What do you think?

中文

谢谢，这是一个很好的讨论，我觉得我们已经很接近了。结合当前实现再看，我在想实现上是否还可以稍微简单一点：

• TurnScope 也许只需要保存已激活 canonical name 的 Set，schema 仍由 bound catalog 统一管理。
• direct/searchable 分类可以只保留一个 catalog 侧的权威来源。
• 搜索索引可以直接从 immutable bound catalog 派生，暂时不引入单独的缓存抽象。
• 旧的 group/economy/replay 状态可以移除，而 load_tools 只保留历史对话的展示兼容。

这样核心流程会比较小：

bound catalog → inventory → tool_search → activated-name set → 下一步投影 → turn 结束时清空

对 v1 来说，我也在想单次搜索限制加 telemetry 是否已经足够。如果真实轨迹之后证明有需要，我们随时可以再增加 turn 级别的总预算。你觉得呢？

Drafted with Codex and reviewed by the author.