The SAFE review framework evaluates each control layer independently: model behavior, instructions, safeguards, tools, environment, monitoring, human operations, and supply chain. Each question asks whether a specific control was operating as intended at the time of the incident.
This structure does not address a category of failure where every individual control passes and the incident emerges from the composition of individually-compliant actions.
The composition problem
An agent sends five queries over twenty minutes, each requesting a small, policy-compliant slice of a dataset. No single query triggers a classification rule. No single response exceeds a data volume threshold. Every individual action is governed, classified, and allowed. The union of the five responses reconstructs a complete dataset that the agent was never authorized to access in full. The exfiltration is invisible to per-action governance because no single action violated policy.
This is not a hypothetical edge case. It is a known attack pattern in database security (query inference attacks) that transfers directly to AI agent workflows. The difference is that AI agents operate at machine speed across multiple tools, providers, and organizational boundaries, making the composition window too narrow for human reviewers to catch in real time.
The composition problem manifests in several forms:
Privilege accumulation. An agent delegates to a second agent, which delegates to a third. Each delegation is authorized within the delegating agent's own scope. But intermediate agents hold independently-granted permissions beyond what they received through delegation, and downstream delegations draw from the full scope of the delegating agent rather than from the scope received from the original principal. Concretely: Agent A has scope {read, write} and delegates {read} to Agent B. Agent B has its own independently-held scope {read, execute}. When B delegates to Agent C, B delegates from its full scope {read, execute}. Agent C now holds {execute}, which A never authorized. Each delegation was valid against the delegating agent's own scope. But the chain produced a privilege set the original principal never granted, because each intermediate agent contributed its own independently-held permissions to the delegation. Per-action governance at each hop sees an authorized delegation. The aggregate privilege trajectory relative to the original principal's authorization is invisible without cross-hop analysis.
Budget and resource exhaustion. An agent makes individually-permitted financial commitments across multiple tools: a cloud compute reservation, a data purchase, a service subscription. Concretely: the agent has a $10,000 authorized budget and a $5,000 per-transaction limit. It commits $4,000 for cloud compute, $3,500 for a data license, and $4,800 for a contractor API. Each transaction clears the per-transaction limit. The total is $12,300, exceeding the authorized budget by 23%. Per-action governance sees three compliant transactions. The aggregate spend is ungoverned because no governance evaluation spans the set of commitments.
Sensitivity escalation across delegation boundaries. An agent operating under a CONFIDENTIAL sensitivity ceiling delegates to a second agent whose own governance permits SECRET. Concretely: Agent A is authorized to handle customer financial records classified as CONFIDENTIAL. It delegates an analysis task to Agent B, a specialized analytics agent whose own governance permits it to operate on SECRET-level data. Agent B processes the CONFIDENTIAL financial records under its SECRET-ceiling environment, which applies SECRET-level retention and logging policies to data that should have been handled under CONFIDENTIAL constraints. Each agent individually complies with its own governance policy. The sensitivity escalation occurs at the delegation boundary, which neither agent's per-action governance evaluates. The delegation infrastructure enforces each agent's own governance policy. It does not evaluate whether the delegation itself moves work to a higher sensitivity ceiling than the delegating agent was operating under. That evaluation requires governance infrastructure at the delegation boundary that treats the delegation as a governed action, distinct from the work that follows it.
Cross-organizational composition. Agent A in Organization 1 writes a partial result to a shared tool. Agent B in Organization 2 reads it, combines it with data from a different source, and produces an output that neither organization's governance policies individually prohibited. Concretely: Organization 1's agent queries a financial database and writes a summary of Q3 revenue by region to a shared workspace. Organization 2's agent reads that summary, combines it with employee headcount data from its own HR system, and calculates per-employee revenue by region. Neither organization's governance prohibited its agent's individual action. The combined output reveals compensation benchmarking data that both organizations classify as restricted. The harmful composition spans two governance instances that have no shared view of the aggregate workflow.
What SAFE's review framework misses
When a SAFE reviewer examines an incident caused by composition, the review framework provides no question that surfaces the composition itself. "Were classifiers, policies, approvals and action limits operating as intended?" The answer is yes: every individual policy evaluation worked correctly. "Could operators detect and interrupt unexpected behavior in real time?" The answer is no, because no individual action was unexpected. "Were responsibilities, escalation paths and kill procedures clear?" Irrelevant: no kill switch condition was met, because each action was compliant.
The incident's cause is not a control failure. It is a gap in the governance architecture: the absence of trajectory analysis across a sequence of individually-governed actions.
Infrastructure requirements for composition governance
Addressing composition requires governance infrastructure that operates above the per-action level. Several structural capabilities are prerequisites.
Workflow-level policy evaluation. Governance infrastructure must evaluate not only whether an individual action complies with policy, but whether the sequence of actions within a workflow complies with aggregate constraints. This requires maintaining workflow state: cumulative data volume accessed, cumulative financial commitments made, cumulative privilege scope granted, and sensitivity ceiling trajectory across delegation hops. Per-action governance produces per-action evidence. Composition governance produces per-workflow evidence that records the aggregate trajectory and the policy evaluation applied to it. The retention window for workflow state is a policy decision for each deployer: a twenty-minute query-inference scenario and a three-week multi-agent research workflow impose different computational and storage requirements. The governance infrastructure must support configurable retention for composition analysis, scoped to the deployer's risk profile.
Cross-agent delegation governance. When one agent delegates to another, the governance infrastructure at the delegation boundary must evaluate and record: what scope was delegated, whether the delegated scope exceeds what the delegating agent was authorized to grant, what sensitivity ceiling applies to the delegated work, and whether revocation of the delegating agent's authority propagates to the delegate. These are governance decisions at the delegation boundary, distinct from the per-action governance each agent receives individually. Without delegation-boundary governance, a SAFE reviewer can see that delegation occurred but cannot determine whether the delegation itself was authorized, appropriately scoped, or properly constrained.
Cross-organizational composition analysis. When a workflow spans multiple governance instances in different organizations, composition analysis requires those instances to share enough workflow context to detect cross-boundary composition threats. This does not require sharing raw data or policy details. It requires governance instances to have enough shared context to evaluate whether a cross-boundary workflow, taken as a whole, remains within the governance constraints each organization imposed on its portion. The specific form of that shared context is an open design question. What matters is the requirement: each governance instance must be able to determine whether the aggregate workflow across the boundary is compliant, without requiring either party to disclose its internal policies or raw data to the other. If this property is unachievable for a given class of cross-boundary workflow, that finding itself is valuable because it identifies a category of composition that is structurally ungovernable under current infrastructure, and that SAFE reviewers should know to flag. Without any cross-boundary context, each governance instance evaluates its own fragment in isolation, and the composition across the boundary is ungoverned.
Revocation propagation across delegation chains. When an agent's authority is revoked mid-workflow (kill switch activation, certification expiry, policy change), the revocation must propagate to every agent working on behalf of the revoked agent. In a delegation chain spanning multiple organizations and multiple governance instances, propagation requires a mechanism that reaches every downstream delegate. Without propagation, a revoked agent's delegated work continues under authority that no longer exists, and the evidence record shows the revocation of the original agent but not the continued operation of its delegates.
Policy reconciliation across governance instances. When a workflow crosses an organizational boundary, the two governance instances may enforce different policies on the same content categories. One instance classifies financial data at a BLOCKED threshold; the other classifies it at REDACTED. When the workflow crosses the boundary, which policy governs? If the receiving instance's policy applies without reconciliation, the boundary crossing effectively downgrades governance. A reconciliation protocol that establishes the effective policy at each boundary, and records the reconciliation decision in the evidence, is required for a SAFE reviewer to determine whether the cross-boundary governance was adequate.
Liability attribution across delegation. When a delegated agent causes harm, the governance evidence must support determining accountability. This requires the evidence record to capture not only that delegation occurred, but who authorized it, what scope was granted, whether the delegate exceeded that scope, and at what point the harm-causing action diverged from the authorized delegation. Per-action evidence from each individual agent is insufficient: the liability question spans the delegation relationship, and the evidence must span it too.
A composition review layer for SAFE
Adding a composition dimension to the SAFE review framework would make these failures reviewable. Possible additions:
Aggregate action sequence compliance. Did aggregate action sequences within a workflow comply with cumulative constraints, or did individually-compliant actions compose into an unauthorized outcome? This surfaces query inference, budget exhaustion, and cumulative data access violations.
Delegation boundary governance. Were delegation boundaries governed: was scope evaluated, was ceiling inheritance enforced, did revocation propagate? This surfaces privilege accumulation, sensitivity escalation, and orphaned delegations.
Cross-organizational policy reconciliation. When the workflow crossed organizational boundaries, was the effective policy at each boundary reconciled and recorded? This surfaces cross-org composition and policy downgrade at boundaries.
Workflow trajectory reconstruction. Can the complete workflow trajectory be reconstructed from evidence produced by all participating governance instances? This surfaces evidence chain gaps, missing delegation context, and incomplete audit trails.
These questions do not replace the existing per-layer review. They add a trajectory dimension that evaluates whether individually-compliant actions composed into an outcome that no individual control was designed to prevent.
Summary
The composition problem is a structural gap in any governance framework that evaluates actions individually. SAFE's review framework inherits this gap from the per-layer control model it applies. Specifying composition governance requirements (workflow-level policy evaluation, delegation-boundary governance, cross-organizational composition analysis, revocation propagation, policy reconciliation, and liability attribution across delegation chains) would close this gap and make composition incidents reviewable within the framework.
The four-layer substrate architecture described at https://scarpprotocol.com/ addresses the structural relationships between identity, cooperation, governance, and settlement that composition governance depends on.
The SAFE review framework evaluates each control layer independently: model behavior, instructions, safeguards, tools, environment, monitoring, human operations, and supply chain. Each question asks whether a specific control was operating as intended at the time of the incident.
This structure does not address a category of failure where every individual control passes and the incident emerges from the composition of individually-compliant actions.
The composition problem
An agent sends five queries over twenty minutes, each requesting a small, policy-compliant slice of a dataset. No single query triggers a classification rule. No single response exceeds a data volume threshold. Every individual action is governed, classified, and allowed. The union of the five responses reconstructs a complete dataset that the agent was never authorized to access in full. The exfiltration is invisible to per-action governance because no single action violated policy.
This is not a hypothetical edge case. It is a known attack pattern in database security (query inference attacks) that transfers directly to AI agent workflows. The difference is that AI agents operate at machine speed across multiple tools, providers, and organizational boundaries, making the composition window too narrow for human reviewers to catch in real time.
The composition problem manifests in several forms:
Privilege accumulation. An agent delegates to a second agent, which delegates to a third. Each delegation is authorized within the delegating agent's own scope. But intermediate agents hold independently-granted permissions beyond what they received through delegation, and downstream delegations draw from the full scope of the delegating agent rather than from the scope received from the original principal. Concretely: Agent A has scope {read, write} and delegates {read} to Agent B. Agent B has its own independently-held scope {read, execute}. When B delegates to Agent C, B delegates from its full scope {read, execute}. Agent C now holds {execute}, which A never authorized. Each delegation was valid against the delegating agent's own scope. But the chain produced a privilege set the original principal never granted, because each intermediate agent contributed its own independently-held permissions to the delegation. Per-action governance at each hop sees an authorized delegation. The aggregate privilege trajectory relative to the original principal's authorization is invisible without cross-hop analysis.
Budget and resource exhaustion. An agent makes individually-permitted financial commitments across multiple tools: a cloud compute reservation, a data purchase, a service subscription. Concretely: the agent has a $10,000 authorized budget and a $5,000 per-transaction limit. It commits $4,000 for cloud compute, $3,500 for a data license, and $4,800 for a contractor API. Each transaction clears the per-transaction limit. The total is $12,300, exceeding the authorized budget by 23%. Per-action governance sees three compliant transactions. The aggregate spend is ungoverned because no governance evaluation spans the set of commitments.
Sensitivity escalation across delegation boundaries. An agent operating under a CONFIDENTIAL sensitivity ceiling delegates to a second agent whose own governance permits SECRET. Concretely: Agent A is authorized to handle customer financial records classified as CONFIDENTIAL. It delegates an analysis task to Agent B, a specialized analytics agent whose own governance permits it to operate on SECRET-level data. Agent B processes the CONFIDENTIAL financial records under its SECRET-ceiling environment, which applies SECRET-level retention and logging policies to data that should have been handled under CONFIDENTIAL constraints. Each agent individually complies with its own governance policy. The sensitivity escalation occurs at the delegation boundary, which neither agent's per-action governance evaluates. The delegation infrastructure enforces each agent's own governance policy. It does not evaluate whether the delegation itself moves work to a higher sensitivity ceiling than the delegating agent was operating under. That evaluation requires governance infrastructure at the delegation boundary that treats the delegation as a governed action, distinct from the work that follows it.
Cross-organizational composition. Agent A in Organization 1 writes a partial result to a shared tool. Agent B in Organization 2 reads it, combines it with data from a different source, and produces an output that neither organization's governance policies individually prohibited. Concretely: Organization 1's agent queries a financial database and writes a summary of Q3 revenue by region to a shared workspace. Organization 2's agent reads that summary, combines it with employee headcount data from its own HR system, and calculates per-employee revenue by region. Neither organization's governance prohibited its agent's individual action. The combined output reveals compensation benchmarking data that both organizations classify as restricted. The harmful composition spans two governance instances that have no shared view of the aggregate workflow.
What SAFE's review framework misses
When a SAFE reviewer examines an incident caused by composition, the review framework provides no question that surfaces the composition itself. "Were classifiers, policies, approvals and action limits operating as intended?" The answer is yes: every individual policy evaluation worked correctly. "Could operators detect and interrupt unexpected behavior in real time?" The answer is no, because no individual action was unexpected. "Were responsibilities, escalation paths and kill procedures clear?" Irrelevant: no kill switch condition was met, because each action was compliant.
The incident's cause is not a control failure. It is a gap in the governance architecture: the absence of trajectory analysis across a sequence of individually-governed actions.
Infrastructure requirements for composition governance
Addressing composition requires governance infrastructure that operates above the per-action level. Several structural capabilities are prerequisites.
Workflow-level policy evaluation. Governance infrastructure must evaluate not only whether an individual action complies with policy, but whether the sequence of actions within a workflow complies with aggregate constraints. This requires maintaining workflow state: cumulative data volume accessed, cumulative financial commitments made, cumulative privilege scope granted, and sensitivity ceiling trajectory across delegation hops. Per-action governance produces per-action evidence. Composition governance produces per-workflow evidence that records the aggregate trajectory and the policy evaluation applied to it. The retention window for workflow state is a policy decision for each deployer: a twenty-minute query-inference scenario and a three-week multi-agent research workflow impose different computational and storage requirements. The governance infrastructure must support configurable retention for composition analysis, scoped to the deployer's risk profile.
Cross-agent delegation governance. When one agent delegates to another, the governance infrastructure at the delegation boundary must evaluate and record: what scope was delegated, whether the delegated scope exceeds what the delegating agent was authorized to grant, what sensitivity ceiling applies to the delegated work, and whether revocation of the delegating agent's authority propagates to the delegate. These are governance decisions at the delegation boundary, distinct from the per-action governance each agent receives individually. Without delegation-boundary governance, a SAFE reviewer can see that delegation occurred but cannot determine whether the delegation itself was authorized, appropriately scoped, or properly constrained.
Cross-organizational composition analysis. When a workflow spans multiple governance instances in different organizations, composition analysis requires those instances to share enough workflow context to detect cross-boundary composition threats. This does not require sharing raw data or policy details. It requires governance instances to have enough shared context to evaluate whether a cross-boundary workflow, taken as a whole, remains within the governance constraints each organization imposed on its portion. The specific form of that shared context is an open design question. What matters is the requirement: each governance instance must be able to determine whether the aggregate workflow across the boundary is compliant, without requiring either party to disclose its internal policies or raw data to the other. If this property is unachievable for a given class of cross-boundary workflow, that finding itself is valuable because it identifies a category of composition that is structurally ungovernable under current infrastructure, and that SAFE reviewers should know to flag. Without any cross-boundary context, each governance instance evaluates its own fragment in isolation, and the composition across the boundary is ungoverned.
Revocation propagation across delegation chains. When an agent's authority is revoked mid-workflow (kill switch activation, certification expiry, policy change), the revocation must propagate to every agent working on behalf of the revoked agent. In a delegation chain spanning multiple organizations and multiple governance instances, propagation requires a mechanism that reaches every downstream delegate. Without propagation, a revoked agent's delegated work continues under authority that no longer exists, and the evidence record shows the revocation of the original agent but not the continued operation of its delegates.
Policy reconciliation across governance instances. When a workflow crosses an organizational boundary, the two governance instances may enforce different policies on the same content categories. One instance classifies financial data at a BLOCKED threshold; the other classifies it at REDACTED. When the workflow crosses the boundary, which policy governs? If the receiving instance's policy applies without reconciliation, the boundary crossing effectively downgrades governance. A reconciliation protocol that establishes the effective policy at each boundary, and records the reconciliation decision in the evidence, is required for a SAFE reviewer to determine whether the cross-boundary governance was adequate.
Liability attribution across delegation. When a delegated agent causes harm, the governance evidence must support determining accountability. This requires the evidence record to capture not only that delegation occurred, but who authorized it, what scope was granted, whether the delegate exceeded that scope, and at what point the harm-causing action diverged from the authorized delegation. Per-action evidence from each individual agent is insufficient: the liability question spans the delegation relationship, and the evidence must span it too.
A composition review layer for SAFE
Adding a composition dimension to the SAFE review framework would make these failures reviewable. Possible additions:
Aggregate action sequence compliance. Did aggregate action sequences within a workflow comply with cumulative constraints, or did individually-compliant actions compose into an unauthorized outcome? This surfaces query inference, budget exhaustion, and cumulative data access violations.
Delegation boundary governance. Were delegation boundaries governed: was scope evaluated, was ceiling inheritance enforced, did revocation propagate? This surfaces privilege accumulation, sensitivity escalation, and orphaned delegations.
Cross-organizational policy reconciliation. When the workflow crossed organizational boundaries, was the effective policy at each boundary reconciled and recorded? This surfaces cross-org composition and policy downgrade at boundaries.
Workflow trajectory reconstruction. Can the complete workflow trajectory be reconstructed from evidence produced by all participating governance instances? This surfaces evidence chain gaps, missing delegation context, and incomplete audit trails.
These questions do not replace the existing per-layer review. They add a trajectory dimension that evaluates whether individually-compliant actions composed into an outcome that no individual control was designed to prevent.
Summary
The composition problem is a structural gap in any governance framework that evaluates actions individually. SAFE's review framework inherits this gap from the per-layer control model it applies. Specifying composition governance requirements (workflow-level policy evaluation, delegation-boundary governance, cross-organizational composition analysis, revocation propagation, policy reconciliation, and liability attribution across delegation chains) would close this gap and make composition incidents reviewable within the framework.
The four-layer substrate architecture described at https://scarpprotocol.com/ addresses the structural relationships between identity, cooperation, governance, and settlement that composition governance depends on.