From 7930b099df1040447af4f44a4a41060256705645 Mon Sep 17 00:00:00 2001 From: Adam Lin Date: Sun, 23 Aug 2026 12:42:32 +0800 Subject: [PATCH] rfc: classify findings and make catalog detection rules executable and testable Signed-off-by: Adam Lin --- rfc-safe-proposal.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/rfc-safe-proposal.md b/rfc-safe-proposal.md index 8a3954f..a421797 100644 --- a/rfc-safe-proposal.md +++ b/rfc-safe-proposal.md @@ -102,6 +102,7 @@ SAFE should adopt the strongest features of confidential safety-reporting system Every incident review should produce shared defensive recommendations that AI providers, deployers, evaluators and customers can implement and verify. Each recommendation should specify: * The failure and affected systems +* The technique classification of the failure against open, vendor-neutral taxonomies (for example MITRE ATLAS techniques or the OWASP LLM and agentic classes), so that recurrence of the same failure across members and time is mechanically detectable rather than editorially asserted * The required defensive outcome * The minimum control and acceptable alternatives * A reproducible verification method @@ -110,6 +111,8 @@ Every incident review should produce shared defensive recommendations that AI pr When doing so does not expose sensitive evidence or create additional risk, SAFE will publish reusable tests, machine-readable policies, detection rules, reference configurations and incident-response guidance. SAFE will maintain these materials in a shared, versioned catalog of incident-driven defensive recommendations. +Detection rules published to the catalog should be expressed in open, executable, vendor-neutral formats rather than prose, and should carry the same discipline the recommendations themselves require: declared true-positive and true-negative test cases, measured false-positive evidence on benign traffic, a version, and the technique classification above. Open rule standards with these properties already exist at both layers of the operating stack — for example Sigma for infrastructure telemetry, and ATR (Agent Threat Rules, MIT) for the agent layer of prompts, tool calls and skill artifacts, which already carries crosswalks to ATLAS and the OWASP classes. A catalog entry that names its technique, ships its tests and states its measured error rates is a control a member can adopt and verify; one that does not is advice. + For unintended access to real systems, recommendations might include default-deny network egress, explicit target allowlists, signed evaluation manifests, independent preflight isolation checks, real-time action monitoring, automatic stops when scope is uncertain and equivalent assurance requirements for evaluation partners. # The Compact