Skip to content

feat(crewai): add CREW-013, ambiguous tool name - #105

Open
bradAGI wants to merge 1 commit into
trustabl:mainfrom
bradAGI:feat/crewai-ambiguous-tool-name
Open

feat(crewai): add CREW-013, ambiguous tool name#105
bradAGI wants to merge 1 commit into
trustabl:mainfrom
bradAGI:feat/crewai-ambiguous-tool-name

Conversation

@bradAGI

@bradAGI bradAGI commented Aug 24, 2026

Copy link
Copy Markdown

Ports the ambiguous-name check to CrewAI. Claude SDK (CSDK-007), OpenAI (OAI-007), ADK (ADK-007), and MCP (MCP-003) all ship it; the newer packs had none.

The tool name sits directly beside the description in what the model sees, so it is half the selection signal — and a generic verb like process or handle spends that half on nothing. In a crew the wrong pick does not stay local: the task output it produces is threaded forward as context to every task behind it, and CREW-104 delegation means peers select from the same name.

Same name_in list as CSDK-007, so behavior stays consistent across packs.

Verification — engine built at main:

$ trustabl rules validate .
OK: 85 rule pack(s), 211 rule(s) valid under rule schema version 14

Fire (tool named process): includes CREW-013
Silent (same tool renamed summarize_invoice): CREW-013 absent

A note on scope. I drafted a fifth rule in this batch for the Vercel AI SDK and dropped it after testing: ToolDef.Name is deliberately empty for vercel_ai_tool, because the SDK derives a tool's name from the key in the tools: { ... } record rather than from the tool({...}) call. The engine asserts this directly —

// internal/analysis/ts_vercel_tools_test.go
if tl.Name != "" {
    t.Errorf("Name: got %q, want empty (Vercel derives name from the tools-record key)", tl.Name)
}

so every name-based predicate (name_in, name_has_prefix) is structurally inert for that pack — it also blocks a Vercel idempotency rule, which is why the pack has none. Back-filling Name from the record key at discovery time would unblock both. Happy to open that engine PR if it'd be useful.

Paired engine PR follows, mirroring the rule into testdata/rules-fixture/ with fire/silent cases.

Ports the ambiguous-name check to crewai. Claude SDK (CSDK-007), OpenAI
(OAI-007), ADK (ADK-007), and MCP (MCP-003) all ship it; the five newer
packs had none.

The tool name sits directly beside the description in what the model sees,
so it is half the selection signal, and a generic verb like process or
handle spends that half on nothing.
@jhumel-code

Copy link
Copy Markdown
Collaborator

Thanks @bradAGI. Clean port, and the note about ToolDef.Name being empty for vercel_ai_tool is exactly the check I want before a name-based rule ships. The rule is schema-clean: crewai_tool is valid at tool scope, name_in exists on main, and low at 0.9 matches CSDK-007.

One thing to resolve first, and it is a close cousin of the Vercel problem you already caught. buildTool sets ToolDef.Name from the Python function name, and toolDecoratorConfig collects only keyword_argument nodes, so a positional decorator argument is dropped. CrewAI's documented form passes the display name positionally, as in @tool("Fetch order status") over def process(...). As written CREW-013 fires there even though the model never sees process, and stays silent on @tool("process") over a well-named function. Capturing that positional name in discovery and matching on it fixes both directions. If you would rather ship narrow first, lower the confidence and say in the explanation that the check reads the Python function name.

Two nits: the last explanation sentence runs to roughly 230 characters on one line where the corpus wraps near 80, and no shipped rule text cites another rule ID, so I would drop the CREW-104 mention.

New rules also need a rationale doc in trustabl-rulebook, which is ours to write and I am happy to pair on. The fixture in trustabl/trustabl#177 matches this file line for line, so the two move together.

Happy to take this once the decorator-name question is settled.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants