Skip to content

feat(langchain): add human approval prerequisite rule - #110

Open
dezsow wants to merge 1 commit into
trustabl:mainfrom
dezsow:hackathon/langchain-hitl-approval
Open

feat(langchain): add human approval prerequisite rule#110
dezsow wants to merge 1 commit into
trustabl:mainfrom
dezsow:hackathon/langchain-hitl-approval

Conversation

@dezsow

@dezsow dezsow commented Aug 24, 2026

Copy link
Copy Markdown

Adds LC-112, a LangChain v1 create_agent rule for privileged execution tools (ShellTool, PythonREPLTool, PythonAstREPLTool) that lack the structural prerequisites for a resumable human-approval boundary.
The rule reuses existing agent predicates—no schema or evaluator changes—and checks for non-empty middleware plus a checkpointer when privileged execution is exposed. Companion changes in trustabl mirror the fixture and add regression coverage; the rulebook PR documents the threat model, calibration, and limitations.

@jhumel-code

Copy link
Copy Markdown
Collaborator

Thanks for this, @dezsow. The rule is schema-clean: every required field is set, langchain_agent is a valid applies_to for agent scope, and all four predicates already exist on main, so no schema_version bump is needed. agent_kwarg_list_empty treats an absent kwarg as empty, so the any branch reads as "middleware not configured or checkpointer not configured", which is what the title claims. The explanation and fix are framed for LangChain rather than borrowed from another SDK, and I appreciate that the fix says out loud that this only verifies structural prerequisites. The fixture mirror in trustabl/trustabl#181 matches this file exactly.

One question before merging. LC-101 in this same pack already fires at high / 0.85 on exactly those three tool classes for langchain_agent, so every LC-112 hit is a second high on the same call site, and LC-101's fix already prescribes the human-in-the-loop gate. Would medium be fairer here? Worth capturing as a limitation too: on LangGraph Platform the checkpointer comes from the deployment and passing one is discouraged, so a correctly gated agent there still trips that branch.

This needs to land with trustabl/trustabl#181, which is missing its policies_test.go coverage entries, so I cannot take it alone yet. LC-112 also needs a rationale doc in trustabl-rulebook; that part is ours and we are glad to pair on it.

Happy to merge once the severity question is settled and the engine side is green.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants