From 9720a5adafa4056d80db22f028e36a0eeece4d76 Mon Sep 17 00:00:00 2001 From: Noetion <294263306+noetion@users.noreply.github.com> Date: Tue, 29 Sep 2026 23:24:14 +0100 Subject: [PATCH] docs: clarify evaluation method and measured limitations --- README.md | 18 ++++++++++++++++-- src/tripwire/reporting.py | 18 ++++++++++++++++-- 2 files changed, 32 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 8d9d886..3ef7c99 100644 --- a/README.md +++ b/README.md @@ -1,8 +1,16 @@ # Tripwire -Tripwire evaluates deterministic sequence rules over AI-agent tool-call traces. Its first measured rule, `TW-001`, looks for an external tool result followed by a sensitive read and then an external write in one session. +**Reproducible evaluation of a sequence rule for AI-agent prompt injection.** -The main finding is a provenance failure, not a new attack signature. Eight of the eleven canonical holdout false negatives received attacker-controlled strings inside transaction records returned by banking tools classified as internal. Tool-result-level provenance is too coarse when a trusted tool returns mixed-trust records; item-level provenance is required to distinguish the injected content without distrusting the entire result. +Tripwire tests whether the order of an agent's tool calls can identify successful prompt-injection outcomes in recorded AgentDojo banking sessions. It combines a sanitised replay corpus, a deterministic rule engine, and a reproducible evaluation with confidence intervals and an error audit. + +Its first measured rule, `TW-001`, looks for this sequence within one session: + +```text +external tool result → sensitive read → external write +``` + +The canonical holdout finds 6 of 17 successful attack sessions, with 1 false positive among 37 negative sessions. The misses expose a practical limit: a banking tool classified as internal can still return attacker-controlled content. The [results](#canonical-combination-holdout) and [error analysis](#why-the-rule-missed-attacks) explain that boundary. Tripwire is a research prototype and reproducibility artifact. It is not affiliated with Fortra's Tripwire products. GitHub issues may be used to report reproducibility defects; no support or release cadence is promised. @@ -11,6 +19,8 @@ Tripwire is a research prototype and reproducibility artifact. It is not affilia Prerequisites: Python 3.11–3.13 and [uv](https://docs.astral.sh/uv/getting-started/installation/). ```console +git clone https://github.com/noetion/tripwire.git +cd tripwire uv sync --frozen uv run tripwire verify ``` @@ -31,6 +41,10 @@ The canonical split hashes `(suite, user task, injection task)` tuples. It measu `TW-001` is now frozen. Its canonical holdout is spent because every error has been inspected. Any successor must be evaluated on a fresh split or corpus; a result produced by revising the rule against this holdout must be labelled contaminated rather than reported as new holdout performance. +## Why the rule missed attacks + +Eight of the eleven canonical holdout false negatives received attacker-controlled strings inside transaction records returned by banking tools classified as internal. Tool-result-level provenance is too coarse when a trusted tool returns mixed-trust records; item-level provenance is required to distinguish the injected content without distrusting the entire result. This is a finding from the [holdout error audit](docs/evidence-audit.md), not a tested successor detector. + ## Injection-family stress test A secondary deterministic split hashes `(suite, injection task)` so complete injection families stay together. It holds out `injection_task_7`, `injection_task_8` and recalls 6/13 (46.2%, Wilson 95%: 23.2%–70.9%). This is numerically higher than the canonical 35.3%, but the intervals overlap substantially, so the numerical difference should not be interpreted as evidence that recall differs. The stress test reports recall only: benign no-injection negatives have no injection family and cannot be assigned by this grouping rule, so precision and false-positive rate would not be comparable. It is a post-hoc stress test rather than an unbiased unseen-attack estimate because the original development partition exposed TW-001's development process to examples from every injection family. diff --git a/src/tripwire/reporting.py b/src/tripwire/reporting.py index cbf228e..c747c3f 100644 --- a/src/tripwire/reporting.py +++ b/src/tripwire/reporting.py @@ -16,9 +16,17 @@ def render_readme(result: dict[str, Any]) -> str: family_names = ", ".join(f"`{name}`" for name in family_holdout["holdout_families"]) return f"""# Tripwire -Tripwire evaluates deterministic sequence rules over AI-agent tool-call traces. Its first measured rule, `TW-001`, looks for an external tool result followed by a sensitive read and then an external write in one session. +**Reproducible evaluation of a sequence rule for AI-agent prompt injection.** -The main finding is a provenance failure, not a new attack signature. Eight of the eleven canonical holdout false negatives received attacker-controlled strings inside transaction records returned by banking tools classified as internal. Tool-result-level provenance is too coarse when a trusted tool returns mixed-trust records; item-level provenance is required to distinguish the injected content without distrusting the entire result. +Tripwire tests whether the order of an agent's tool calls can identify successful prompt-injection outcomes in recorded AgentDojo banking sessions. It combines a sanitised replay corpus, a deterministic rule engine, and a reproducible evaluation with confidence intervals and an error audit. + +Its first measured rule, `TW-001`, looks for this sequence within one session: + +```text +external tool result → sensitive read → external write +``` + +The canonical holdout finds {recall["numerator"]} of {recall["denominator"]} successful attack sessions, with {counts["false_positives"]} false positive among {counts["negative"]} negative sessions. The misses expose a practical limit: a banking tool classified as internal can still return attacker-controlled content. The [results](#canonical-combination-holdout) and [error analysis](#why-the-rule-missed-attacks) explain that boundary. Tripwire is a research prototype and reproducibility artifact. It is not affiliated with Fortra's Tripwire products. GitHub issues may be used to report reproducibility defects; no support or release cadence is promised. @@ -27,6 +35,8 @@ def render_readme(result: dict[str, Any]) -> str: Prerequisites: Python 3.11–3.13 and [uv](https://docs.astral.sh/uv/getting-started/installation/). ```console +git clone https://github.com/noetion/tripwire.git +cd tripwire uv sync --frozen uv run tripwire verify ``` @@ -47,6 +57,10 @@ def render_readme(result: dict[str, Any]) -> str: `TW-001` is now frozen. Its canonical holdout is spent because every error has been inspected. Any successor must be evaluated on a fresh split or corpus; a result produced by revising the rule against this holdout must be labelled contaminated rather than reported as new holdout performance. +## Why the rule missed attacks + +Eight of the eleven canonical holdout false negatives received attacker-controlled strings inside transaction records returned by banking tools classified as internal. Tool-result-level provenance is too coarse when a trusted tool returns mixed-trust records; item-level provenance is required to distinguish the injected content without distrusting the entire result. This is a finding from the [holdout error audit](docs/evidence-audit.md), not a tested successor detector. + ## Injection-family stress test A secondary deterministic split hashes `(suite, injection task)` so complete injection families stay together. It holds out {family_names} and recalls {family_recall["numerator"]}/{family_recall["denominator"]} ({_percent(family_recall["value"])}, Wilson 95%: {_interval(family_recall["wilson_95"])}). This is numerically higher than the canonical 35.3%, but the intervals overlap substantially, so the numerical difference should not be interpreted as evidence that recall differs. The stress test reports recall only: benign no-injection negatives have no injection family and cannot be assigned by this grouping rule, so precision and false-positive rate would not be comparable. It is a post-hoc stress test rather than an unbiased unseen-attack estimate because the original development partition exposed TW-001's development process to examples from every injection family.