Add first-party harm and acts of omission to the SAFE reporting criteria

[ballen-m07]

0. The Ask

Today's version of the draft contains reporting criteria for AI acts of commission that affect a third party. It should be expanded to include reporting criteria for incidents where AI causes first-party damage (internal to the organization). Additionally, the reporting criteria should expand to include acts of omission related to a security control that cause damages or near misses (either internal or external).

---

1. The Need

Frameworks like the SAFE framework are critical to community learning. Artificial Intelligence presents new risks that people are unfamiliar with. Traditional automations are deterministic and fail in deterministic ways. AI introduces non-deterministic mechanisms into automation pipelines. As organizations adopt AI into their various pipelines (both security pipelines and operational pipelines), the opportunities for drift (slight deviations from expected behavior) increase. The more layered the pipeline, the greater the risk of drift accumulation. In addition to these risks, the scope of action (or inaction) potentials is significantly larger than what people are used to developing constraints over. The community at large is learning how to properly scope AI instructions and bounds. Frameworks like this are invaluable for increasing the rate at which the community learns. In establishing robust criteria for reporting, we give others the opportunity to learn from reported incidents.

---

2. Finding

The current draft provides the following four reportable events:

• Accesses, exploits, disrupts, misuses, or modifies a third-party system without authorization.
• Causes third-party impact by escaping or bypassing a sandbox, network, identity, policy or tool boundary.
• Accesses third party confidential information, for example, by accessing data, or redistributing for others to access, without consent of the owner.
• Continues to probe, access, exploit, or modify a production target after the operator knows or reasonably suspects that the activity is unauthorized or outside the approved scope.

Current language is framed towards acts of commission that affect third parties, which is one of four domains that should be covered in this framework.

Coverage:

	Acts	Omissions
Third-party harm	covered	not covered
First-party harm	not covered	not covered

---

3. Why

Expanding the scope to include the other three domains increases the opportunities for the community to learn. Especially given that the majority of AI activity occurs within trust boundaries. First-party reporting is the norm across compliance frameworks (FISMA, SEC, NYDFS, etc.). As it stands, the SAFE framework is an outlier. Excluding (or not highlighting and specifically calling out) first-party reporting leaves significant learning opportunities on the table.

Additionally, acts of omission are likely to be pervasive as they can fail silently. A pervasive issue risks stacking up impacts until noticed. Validators and safeguards mitigate against unexpected, damaging actions (or inactions); however, the non-deterministic nature of AI agents in increasingly complicated pipelines with multiple agents working serially and/or in parallel introduce their own complexities where drift or inference of safety guardrails from one agent could prevent an expected action. Near misses are included in the current version of the draft and the expanded scope proposed here is a rich ground for near-miss learning opportunities.

---

4. Proposed changes

New reportable-event clause.

• System failed to take an expected action related to a security control that results in damages (or near misses) internally or externally, including failures due to a constraint inferred by the agent.

New Observability requirement.

• Operators should declare expected actions for agents along the pipeline in order to ensure that acts of omission are observable and reportable.

Clarification for bullet four.

• Continues to probe, access, exploit, or modify a production target (first- or third-party) after the operator knows or reasonably suspects that the activity is unauthorized or outside the approved scope.

---

5. What this comment does not ask for

This comment is not asking to turn the SAFE framework into some kind of reliability framework, but instead aims at ensuring that security incidents across all four domains are considered for reporting to ensure learning across the community. The ask is for reporting mechanisms regarding omissions that are related to security controls.

---

6. Relationship to existing issues

This proposal operates on the reporting criteria. The open issues I read all work downstream of that section (evidence format, evidence integrity, verification-method quality, system identifiers, causal structure), so I don't believe it duplicates any of them. Where it touches existing work:

• Add Temporal Authority and Self-Extension Denial to the SAFE Evidence Framework #1 (Temporal Authority and Self-Extension Denial). Different vector, despite similar language. Self-extension is an agent expanding authority beyond what was granted. The inferred-constraint case here is the inverse: an agent declining authority it legitimately has, because it adopted a rule nobody gave it.
• Add an optional causal incident graph profile for reconstructable, mechanically verifiable SAFE findings #7 (optional causal incident graph profile). Complementary. That profile already declares invariants and evaluates them as held or violated, and already targets near misses. An expected action that did not occur would be a natural sibling to an invariant, and the omission clause proposed here would slot into that graph directly.
• Evidence Preservation specifies what to retain but not that it be independently verifiable #11 (evidence preservation and independent verifiability). Observes that when an incident involves a member's own AI system in the member's own environment, the analysis rests substantially on that member's own account. That assumes first-party incidents are in scope. The reporting criteria as written do not grant it.
• Verification methods should disclose systematic mislabel classes (well-formed wrong answers), not only fail-open and noise #10 (systematic mislabel classes). Measures the silent-failure problem directly: judges scored a transcript as compliant when the agent completed a harmless partial step and refused the operative instruction. That is the failure mode this proposal wants reportable.
• Support design-time control lineage in SAFE findings (design_coverage + design_references) #3 (design_coverage) and Define machine-verifiable approval-to-execution scope binding for SAFE evidence #13 (approval-to-execution scope binding). Both already build declaration machinery. The observability requirement proposed here should reuse those rather than introduce a parallel mechanism.
• Require a declared failure mode and a stated noise floor for verification methods in SAFE recommendations #4 (declared failure mode and noise floor). Related terminology, different subject. That declaration is about whether a verification method actually measured. This one is about whether the agent actually acted.