Skip to content

test(langchain): add LC-103 fixture coverage - #175

Open
jiaxinaspenlin-dotcom wants to merge 1 commit into
trustabl:mainfrom
jiaxinaspenlin-dotcom:feat/langchain-requests-builtin-tool
Open

test(langchain): add LC-103 fixture coverage#175
jiaxinaspenlin-dotcom wants to merge 1 commit into
trustabl:mainfrom
jiaxinaspenlin-dotcom:feat/langchain-requests-builtin-tool

Conversation

@jiaxinaspenlin-dotcom

Copy link
Copy Markdown

Summary

Adds the engine-side fixture and test coverage for LC-103, which detects LangChain agents that wire langchain_community's Requests* built-in HTTP tools.

What changed

  • Mirrored LC-103 into:
    • testdata/rules-fixture/langchain/agent_safety.yaml
  • Added three agent-rule test cases in:
    • internal/rules/policies_test.go

Coverage includes:

  • FIRE: ReactAgent with RequestsGetTool
  • FIRE: AgentExecutor with RequestsPostTool
  • SILENT: ReactAgent with TavilySearchResults

Every AgentDef sets Language.

No analyzer changes were required because the existing LangChain hosted-tool discovery already recognizes all five Requests* classes.

Validation

  • go vet ./... — pass
  • go test -race ./internal/rules/... — pass
  • go test -race ./... — 26 packages passed, 0 failures
  • go build -o /tmp/trustabl ./cmd/trustabl — pass
  • check-rules-sync.sh — in sync with production

The fixture is byte-identical to the corresponding rule definition in trustabl-rules.

Paired changes

All coordinated changes use the shared branch:

feat/langchain-requests-builtin-tool

@jhumel-code

Copy link
Copy Markdown
Collaborator

Thanks @jiaxinaspenlin-dotcom, this is a clean engine half. I compared the testdata/rules-fixture/langchain/agent_safety.yaml hunk against the one in trustabl/trustabl-rules#103 and it matches line for line, so rules-sync stays green provided the two land together. The three cases follow the LC-101 pattern directly above them, set Language, and the two fire cases exercise different applies_to kinds (ReactAgent resolves to langchain_agent, AgentExecutor to langchain_agent_executor), which is more than the minimum the coverage guard asks for.

Two nits, neither blocking. applies_to also lists langchain_state_graph and nothing covers it, so mirroring LC-101's StateGraph case would close that out. The silent case uses TavilySearchResults, which is not in LangChainHostedToolClasses and so is never emitted as a HostedToolRef by a real scan; it does exercise the class filter, but an agent with no hosted refs would be the shape users actually hit.

Coordination note: LC-103 is also claimed by #183 and #186, so whichever lands first, the others will need to renumber and rebase. That is not a mark against this one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants