Skip to content

feat(langchain): audit the requests built-ins, add PythonREPL to LC-101 - #108

Open
ShonenAsh wants to merge 1 commit into
trustabl:mainfrom
ShonenAsh:feat/langchain-http-builtin-coverage
Open

feat(langchain): audit the requests built-ins, add PythonREPL to LC-101#108
ShonenAsh wants to merge 1 commit into
trustabl:mainfrom
ShonenAsh:feat/langchain-http-builtin-coverage

Conversation

@ShonenAsh

Copy link
Copy Markdown

Discovery recognizes nine high-risk LangChain built-in classes (LangChainHostedToolClasses) and emits each as a HostedToolDef edge, but LC-101 consumed only three of them, so six were discovered and never evaluated.

  • LC-101 now also lists the bare PythonREPL utility. It sits in the engine's code-execution set alongside PythonREPLTool, and the rule's own title already covers it, so tools=[PythonREPL()] scanning clean was a false negative on an RCE-class wiring.
  • LC-103 (medium, 0.7) covers RequestsGetTool: a model-chosen destination for an outbound GET, i.e. SSRF plus untrusted-content intake.
  • LC-104 (high, 0.75) covers RequestsPostTool / RequestsPutTool / RequestsPatchTool / RequestsDeleteTool. Destination and body are both model-controlled, so a prompt injection mutates remote state rather than only reading it, and the effect is not undone by discarding the response.

The read/write split is deliberate: a GET and a DELETE against an attacker-chosen host are not the same exposure. Severities are calibrated against the CrewAI analogues for the identical predicate shape, CREW-107 (model-chosen URLs, medium) and CREW-109 (model-driven writes, high).

No new predicates and no schema_version bump --
agent_uses_hosted_tool_class already backs LC-101.

Discovery recognizes nine high-risk LangChain built-in classes
(LangChainHostedToolClasses) and emits each as a HostedToolDef edge, but
LC-101 consumed only three of them, so six were discovered and never
evaluated.

- LC-101 now also lists the bare PythonREPL utility. It sits in the
  engine's code-execution set alongside PythonREPLTool, and the rule's own
  title already covers it, so tools=[PythonREPL()] scanning clean was a
  false negative on an RCE-class wiring.
- LC-103 (medium, 0.7) covers RequestsGetTool: a model-chosen destination
  for an outbound GET, i.e. SSRF plus untrusted-content intake.
- LC-104 (high, 0.75) covers RequestsPostTool / RequestsPutTool /
  RequestsPatchTool / RequestsDeleteTool. Destination and body are both
  model-controlled, so a prompt injection mutates remote state rather than
  only reading it, and the effect is not undone by discarding the response.

The read/write split is deliberate: a GET and a DELETE against an
attacker-chosen host are not the same exposure. Severities are calibrated
against the CrewAI analogues for the identical predicate shape, CREW-107
(model-chosen URLs, medium) and CREW-109 (model-driven writes, high).

No new predicates and no schema_version bump --
agent_uses_hosted_tool_class already backs LC-101.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@ShonenAsh

Copy link
Copy Markdown
Author

Heads-up: this overlaps #103, which claims LC-103 for the same Requests* family and was opened first. Two notes on how to untangle it.

The LC-101 change here is independent of that overlap, it adds the bare PythonREPL utility, which is already in LangChainHostedToolClasses as a code-execution class but was unlisted by the rule, so tools=[PythonREPL()] scans clean today. No other open PR touches LC-101. Happy to split that into its own PR so it can land regardless of how the Requests question resolves.

Overlap: #103 has the earlier claim on LC-103 and I'm happy to defer. The one thing I'd argue for keeping is the read/write split: #103's own explanation notes the write verbs "can mutate those services rather than only read them," but rates all five at medium. A model-chosen GET leaks something you can rotate; a model-chosen DELETE has already landed. If useful I can withdraw this and open a follow-up on top of #103 that narrows LC-103 to RequestsGetTool and adds LC-104 (high) for the write verbs.

@jhumel-code

Copy link
Copy Markdown
Collaborator

Thanks @ShonenAsh, this is careful work. I checked LangChainHostedToolClasses on main and the gap you describe is real: the bare PythonREPL has been discovered and emitted as a hosted edge all along, and LC-101 never listed it, so tools=[PythonREPL()] scanned clean. Splitting the requests built-ins by effect is the right call, and your severity and confidence land exactly on CREW-107 (medium, 0.7) and CREW-109 (high, 0.75), which is the calibration I would have asked for. The schema side is clean too: agent_uses_hosted_tool_class already exists on main, all three applies_to tokens are valid at agent scope, and no schema_version bump is warranted. The fixture mirror in trustabl/trustabl#183 is byte identical to this file, so the pair holds together.

Two coordination notes. LC-103 is also claimed by #103 and #113 (engine halves trustabl/trustabl#175 and trustabl/trustabl#186), so whichever lands first, the rest will need to renumber and rebase. Sequencing that is on us, not on you. LC-103 and LC-104 also need rationale docs in trustabl-rulebook before they ship, and we can handle or pair on that part.

One follow-up thought, not for this PR: the common docs idiom is RequestsToolkit(...).get_tools(), which discovery does not see today because it only classifies direct calls inside tools=[...]. Worth widening separately.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants