diff --git a/README.md b/README.md index 8895248..ff3a411 100644 --- a/README.md +++ b/README.md @@ -40,8 +40,9 @@ This result classifies successful prompt-injection outcomes in the committed Age ## Related work - [AgentDojo](https://proceedings.neurips.cc/paper_files/paper/2024/hash/97091a5177d8dc64b1da8bf3e1f6fb54-Abstract-Datasets_and_Benchmarks_Track.html) supplies the agent tasks, attacks, benchmark outcomes and source traces used by the committed corpus. Tripwire adds a sanitised replay corpus and detection-oriented measurement rather than a new attack benchmark. -- [Agent Threat Rules (ATR)](https://github.com/Agent-Threat-Rule/agent-threat-rules) standardises portable rules evaluated against individual agent events and content fields. Tripwire instead measures ordered cross-event behaviour and exposes where event-level tool provenance loses mixed-trust structure inside a result. -- [AgentSigma](https://github.com/cveye/agentsigma) is a close detection-engineering precedent: a Sigma-compatible runtime framework for the agent tool-call layer. Tripwire's narrower contribution is a frozen labelled evaluation, explicit holdout discipline and measured provenance failure. +- [Agent Threat Rules (ATR)](https://github.com/Agent-Threat-Rule/agent-threat-rules) standardises portable detections over agent events and content fields. +- Cross-event agent detection is established prior art, not Tripwire's novelty. [AgentSigma](https://github.com/cveye/agentsigma) applies Sigma-compatible runtime detection at the tool-call layer; [AgentShield's sigma-ai](https://github.com/agentshield-ai/sigma-ai) documents temporal correlation for sequential agent events; [AIDR Sigma](https://github.com/netzilo/aidr-sigma) describes a persistent per-session behaviour graph for multi-step patterns; and [RSigma](https://github.com/timescale/rsigma) provides in-process stateful correlation and OTLP ingest. RSigma is also a candidate substrate for any future streaming backend and should be evaluated before Tripwire implements those capabilities itself. +- Tripwire's contribution here is the measurement: a sanitised labelled AgentDojo corpus, reproducible metrics, a frozen holdout and the provenance-granularity finding that resulted from error analysis. - [Out-of-band agent defenses](https://arxiv.org/abs/2606.26479) use deterministic policies, capabilities or information-flow labels outside the model. Tripwire's finding identifies a boundary condition for designs that collapse provenance to tool identity or whole tool results: a trusted tool can return attacker-controlled fields. ## Evidence diff --git a/src/tripwire/reporting.py b/src/tripwire/reporting.py index aa94e0e..969f582 100644 --- a/src/tripwire/reporting.py +++ b/src/tripwire/reporting.py @@ -56,8 +56,9 @@ def render_readme(result: dict[str, Any]) -> str: ## Related work - [AgentDojo](https://proceedings.neurips.cc/paper_files/paper/2024/hash/97091a5177d8dc64b1da8bf3e1f6fb54-Abstract-Datasets_and_Benchmarks_Track.html) supplies the agent tasks, attacks, benchmark outcomes and source traces used by the committed corpus. Tripwire adds a sanitised replay corpus and detection-oriented measurement rather than a new attack benchmark. -- [Agent Threat Rules (ATR)](https://github.com/Agent-Threat-Rule/agent-threat-rules) standardises portable rules evaluated against individual agent events and content fields. Tripwire instead measures ordered cross-event behaviour and exposes where event-level tool provenance loses mixed-trust structure inside a result. -- [AgentSigma](https://github.com/cveye/agentsigma) is a close detection-engineering precedent: a Sigma-compatible runtime framework for the agent tool-call layer. Tripwire's narrower contribution is a frozen labelled evaluation, explicit holdout discipline and measured provenance failure. +- [Agent Threat Rules (ATR)](https://github.com/Agent-Threat-Rule/agent-threat-rules) standardises portable detections over agent events and content fields. +- Cross-event agent detection is established prior art, not Tripwire's novelty. [AgentSigma](https://github.com/cveye/agentsigma) applies Sigma-compatible runtime detection at the tool-call layer; [AgentShield's sigma-ai](https://github.com/agentshield-ai/sigma-ai) documents temporal correlation for sequential agent events; [AIDR Sigma](https://github.com/netzilo/aidr-sigma) describes a persistent per-session behaviour graph for multi-step patterns; and [RSigma](https://github.com/timescale/rsigma) provides in-process stateful correlation and OTLP ingest. RSigma is also a candidate substrate for any future streaming backend and should be evaluated before Tripwire implements those capabilities itself. +- Tripwire's contribution here is the measurement: a sanitised labelled AgentDojo corpus, reproducible metrics, a frozen holdout and the provenance-granularity finding that resulted from error analysis. - [Out-of-band agent defenses](https://arxiv.org/abs/2606.26479) use deterministic policies, capabilities or information-flow labels outside the model. Tripwire's finding identifies a boundary condition for designs that collapse provenance to tool identity or whole tool results: a trusted tool can return attacker-controlled fields. ## Evidence diff --git a/tests/test_m1_acceptance.py b/tests/test_m1_acceptance.py index 9ffaabd..fe0fc44 100644 --- a/tests/test_m1_acceptance.py +++ b/tests/test_m1_acceptance.py @@ -379,6 +379,11 @@ def test_public_report_preserves_evaluation_discipline() -> None: assert "intervals overlap substantially" in readme assert "reports recall only" in readme assert "## Related work" in readme + assert "Cross-event agent detection is established prior art" in readme + assert "github.com/agentshield-ai/sigma-ai" in readme + assert "github.com/netzilo/aidr-sigma" in readme + assert "github.com/timescale/rsigma" in readme + assert "Tripwire's contribution here is the measurement" in readme assert any("holdout is spent" in limitation for limitation in result["limitations"])