Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 9 additions & 4 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

### Fixed
- **npm CLI accepts a bare spec path** — `oa validate <spec.yaml>` and `oa run <spec.yaml>` now work without `--spec` in the npm runtime, matching the Python CLI (1.6.0). Same guardrails: `--spec` unchanged, bare path + `--spec` together is an explicit error, and a non-YAML bare argument gets a clear error naming the valid forms. First Jest tests land with this (`npm/tests/`), and both CI and the npm publish workflow now run them. (#100)

### Added (older, pre-1.4 notes)
- This changelog.
- **Agents-as-code documentation** — new section in REFERENCE.md explaining the `.agents/` pattern, bundled examples table, and scaffold/run/generate workflows.
Expand All @@ -25,14 +22,22 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Fixed (older, pre-1.4 notes)
- Removed broken references to non-existent `security-threat-analyzer.yaml` template and `SECURITY_TEMPLATES.md` from REFERENCE.md.

## [1.6.1] - 2026-08-31
## [1.6.1] - Unreleased

### Security
- **Declared behavioural contracts now fail closed** — if a resolved task declares a contract but the `behavioural-contracts` enforcement dependency is unavailable, execution stops before the affected model call with `CONTRACTS_UNAVAILABLE` instead of logging a warning and continuing without the promised constraint. Direct dependencies and locally delegated tasks are preflighted before their chain starts. (#103)
- **Sandbox domain rules cover ports and MCP endpoints** — `http.allow_domains` entries may pin `host:port` while bare hosts retain backwards-compatible any-port semantics. Statically configured MCP endpoints are preflighted across the selected task and direct dependencies before discovery or model execution. Malformed allowlist entries now fail validation. (#104)
- **Declared remote delegated-spec URLs respect the delegating task's sandbox** — `http://`, `https://`, and resolved `oa://` destinations are checked against that task's effective `sandbox.http.allow_domains` before the initial request. Redirect destinations remain tracked in #114; cross-document sandbox inheritance remains tracked in #110. (#112)

### Fixed
- **npm CLI accepts a bare spec path** — `oa validate <spec.yaml>` and `oa run <spec.yaml>` now work without `--spec` in the npm runtime, matching the Python CLI (1.6.0). Same guardrails: `--spec` unchanged, bare path + `--spec` together is an explicit error, and a non-YAML bare argument gets a clear error naming the valid forms. First Jest tests land with this (`npm/tests/`), and both CI and the npm publish workflow now run them. (#100)

### Documentation
- Stamp the OA 1.6 normative document and schema metadata as 1.6.1, add revision history, and narrow the compatibility statement to acknowledge stricter `allow_domains` validation. (#111)

### Breaking
- Existing specs that combine MCP tools with `sandbox.http.allow_domains` must add each MCP endpoint host (or `host:port`) to the effective allowlist. Runtimes now enforce the declared network boundary for MCP instead of limiting it to native HTTP tools.
- Existing specs that combine `oa://` delegation with `sandbox.http.allow_domains` must add `openagentspec.dev` to the delegating task's effective allowlist so the registry URL can be fetched.
- Specs that declare `behavioural_contract` must install `open-agent-spec[contracts]`; execution no longer continues without enforcement.

## [1.6.0] - 2026-07-28
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -619,7 +619,7 @@ The formal specification defines what a conforming OA runtime must do, independe

| Resource | Contents |
|----------|----------|
| [spec/open-agent-spec-1.6.md](spec/open-agent-spec-1.6.md) | Formal specification — normative MUST/SHOULD/MAY requirements for OA 1.6.0 |
| [spec/open-agent-spec-1.6.md](spec/open-agent-spec-1.6.md) | Formal specification — normative MUST/SHOULD/MAY requirements for OA 1.6.1 |
| [spec/schema/oas-schema-1.6.json](spec/schema/oas-schema-1.6.json) | Canonical JSON Schema for validating spec documents |
| [spec/conformance/README.md](spec/conformance/README.md) | Conformance test structure and contribution guide |
| [spec/conformance/PROTOCOL.md](spec/conformance/PROTOCOL.md) | Runtime-agnostic adapter protocol — certify any runtime, in any language |
Expand Down
4 changes: 2 additions & 2 deletions docs/REFERENCE.md
Original file line number Diff line number Diff line change
Expand Up @@ -598,12 +598,12 @@ All sandbox violations raise `OARunError` immediately with one of three structur
| Code | Trigger |
|------|---------|
| `SANDBOX_TOOL_VIOLATION` | Tool name not in `allow` list, or in `deny` list |
| `SANDBOX_DOMAIN_VIOLATION` | HTTP or MCP destination not in `allow_domains`; `host:port` rules require that port |
| `SANDBOX_DOMAIN_VIOLATION` | HTTP, MCP, or declared/resolved remote delegated-spec destination not in `allow_domains`; `host:port` rules require that port |
| `SANDBOX_PATH_VIOLATION` | File path outside `allow_paths` (for `file.read` / `file.write`) |

Path traversal (`../../`) is caught automatically — paths are resolved to absolute before comparison.

MCP endpoints are static spec configuration and are checked against the effective `http.allow_domains` policy before tool discovery or model execution. A bare hostname preserves the original any-port behaviour; use `host:port` when the agent must reach only one service on that host.
MCP endpoints and declared remote delegated-spec URLs are static spec configuration and are checked against the current task's effective `http.allow_domains` policy before discovery, the initial fetch, or model execution. For `oa://` references, allow the resolved registry host (`openagentspec.dev`). Redirect destinations and sandbox inheritance across delegated documents are separate concerns. A bare hostname preserves the original any-port behaviour; use `host:port` when the agent must reach only one service on that host.

### Input immutability

Expand Down
24 changes: 22 additions & 2 deletions oas_cli/runner.py
Original file line number Diff line number Diff line change
Expand Up @@ -493,8 +493,9 @@ def _preflight_runtime_guards(

The selected task and its direct dependencies are checked together. Local
delegated specs are recursively inspectable and are therefore included in
the same preflight. Remote specs are guarded after fetch at their execution
boundary because their contents are not locally available.
the same preflight. Remote delegation destinations are checked before
fetch; guards requiring remote contents run after fetch at the execution
boundary because those contents are not locally available.
"""
tasks = spec_data.get("tasks") or {}
task_def = tasks.get(task_name) or {}
Expand All @@ -509,6 +510,7 @@ def _preflight_runtime_guards(
continue
raw_ref = delegation_ref.strip()
if _is_remote_ref(raw_ref):
_check_remote_spec_endpoint(raw_ref, sandbox, resolved_task)
continue

delegated_path = Path(raw_ref)
Expand Down Expand Up @@ -670,6 +672,22 @@ def _check_mcp_endpoints(
)


def _check_remote_spec_endpoint(
ref: str, sandbox: dict[str, Any], task_name: str
) -> None:
"""Preflight a remote delegated-spec destination before network access."""
url = _resolve_spec_url(ref)
allow_domains = (sandbox.get("http") or {}).get("allow_domains")
if allow_domains is None:
Comment thread
aswhitehouse marked this conversation as resolved.
return
_check_url_domain(
url,
allow_domains,
task_name,
source="Delegated spec",
)


_MAX_TOOL_ITERATIONS = 10


Expand Down Expand Up @@ -853,6 +871,8 @@ def _run_single_task(

# ── Remote spec (oa:// or https://) ──────────────────────────────
if _is_remote_ref(raw_ref):
sandbox = _resolve_sandbox(spec_data, task_name)
_check_remote_spec_endpoint(raw_ref, sandbox, task_name)
url = _resolve_spec_url(raw_ref)
# Use URL string as the cycle-detection key.
canonical_key: Any = url
Expand Down
6 changes: 3 additions & 3 deletions oas_cli/schemas/oas-schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"$schema": "http://json-schema.org/draft-07/schema#",
"$id": "https://openagents.org/schemas/oas-schema.json",
"title": "Open Agent Spec",
"description": "Schema for Open Agent Spec YAML files (OA 1.6.0)",
"description": "Schema for Open Agent Spec YAML files (OA 1.6.1)",
"type": "object",
"required": [
"open_agent_spec",
Expand Down Expand Up @@ -589,10 +589,10 @@
"type": "string",
"pattern": "^(?![A-Za-z][A-Za-z0-9+.-]*://)(?:\\[[^\\]]+\\]|[^:/\\s]+)(?::[0-9]{1,5})?$"
},
"description": "Hosts or host:port destinations permitted for http.get/http.post and MCP endpoints. A destination host must equal a listed domain or be a subdomain; an optional port must match the effective destination port. Otherwise SANDBOX_DOMAIN_VIOLATION."
"description": "Hosts or host:port destinations permitted for http.get/http.post, MCP endpoints, and declared or resolved remote delegated-spec URLs. A destination host must equal a listed domain or be a subdomain; an optional port must match the effective destination port. Otherwise SANDBOX_DOMAIN_VIOLATION."
}
},
"description": "Network destination constraints for native HTTP tools and MCP endpoints."
"description": "Network destination constraints for native HTTP tools, MCP endpoints, and declared or resolved remote delegated-spec URLs."
},
"file": {
"type": "object",
Expand Down
12 changes: 9 additions & 3 deletions spec/conformance/CONFORMANCE.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
# OA Conformance Matrix

| Case | python-reference 1.5.2 | npm 1.5.2 |
| Case | python-reference 1.6.1 | npm 1.6.1 |
|---|---|---|
| schema/invalid-allow-domain | ✅ PASS | ✅ PASS |
| schema/invalid-engine | ✅ PASS | ✅ PASS |
| schema/invalid-version | ✅ PASS | ✅ PASS |
| schema/missing-agent | ✅ PASS | ✅ PASS |
Expand Down Expand Up @@ -33,14 +34,19 @@
| errors/chain-cycle | ✅ PASS | ✅ PASS |
| errors/chain-input-missing | ✅ PASS | ✅ PASS |
| errors/contract-violation | ✅ PASS | ⬜ UNSUPPORTED |
| errors/contracts-unavailable | ⬜ UNSUPPORTED | ✅ PASS |
| errors/error-structure | ✅ PASS | ✅ PASS |
| errors/task-not-found | ✅ PASS | ✅ PASS |
| sandbox/domain-port-mismatch | ✅ PASS | ⬜ UNSUPPORTED |
| sandbox/mcp-domain-preflight | ✅ PASS | ⬜ UNSUPPORTED |
| sandbox/oa-registry-domain-preflight | ✅ PASS | ⬜ UNSUPPORTED |
| sandbox/remote-delegation-preflight | ✅ PASS | ⬜ UNSUPPORTED |

## Summary

| Runtime | Pass | Fail | Unsupported | Adapter errors |
|---|---|---|---|---|
| python-reference 1.5.2 | 33 | 0 | 0 | 0 |
| npm 1.5.2 | 31 | 0 | 2 | 0 |
| python-reference 1.6.1 | 38 | 0 | 1 | 0 |
| npm 1.6.1 | 33 | 0 | 6 | 0 |

Legend: ✅ PASS · ❌ FAIL · ⬜ UNSUPPORTED (capability not declared) · 💥 adapter error
4 changes: 2 additions & 2 deletions spec/conformance/README.md
Comment thread
aswhitehouse marked this conversation as resolved.
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
# OA Conformance Tests

This directory contains the conformance test suite for Open Agent Spec 1.6.0. Conformance tests validate **runtime behaviour**, not LLM output.
This directory contains the conformance test suite for Open Agent Spec 1.6.1. Conformance tests validate **runtime behaviour**, not LLM output.

The suite is **runtime-agnostic**: a single harness drives the YAML cases against any OA runtime through a thin subprocess adapter (JSON over stdin/stdout). The protocol is defined in [PROTOCOL.md](PROTOCOL.md). Reference adapters for the Python and npm runtimes live in `adapters/`.

## Purpose

The spec at `../open-agent-spec-1.6.md` defines what a conforming runtime MUST do. These tests operationalise that definition — any runtime that passes the full suite can claim OA 1.6.0 conformance.
The spec at `../open-agent-spec-1.6.md` defines what a conforming runtime MUST do. These tests operationalise that definition — any runtime that passes the full suite can claim OA 1.6.1 conformance.

Note: individual cases embed the **minimum** `open_agent_spec` version their behaviour requires (many say `"1.5.0"` or lower), not the suite version. This is deliberate — the additive-compatibility guarantee means a 1.6-conforming runtime must accept those documents unchanged, and the pinned versions exercise exactly that.

Expand Down
2 changes: 1 addition & 1 deletion spec/conformance/cases/sandbox/domain-port-mismatch.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ spec_section: "11.2"
requires: sandbox

spec: |
open_agent_spec: "1.6.0"
open_agent_spec: "1.6.1"
agent:
name: test
description: test
Expand Down
2 changes: 1 addition & 1 deletion spec/conformance/cases/sandbox/mcp-domain-preflight.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ spec_section: "11.3"
requires: sandbox

spec: |
open_agent_spec: "1.6.0"
open_agent_spec: "1.6.1"
agent:
name: test
description: test
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# Spec §11.2/§11.3 — oa:// references check their resolved registry destination.
description: "A blocked oa registry destination fails during chain preflight"
spec_section: "11.2, 11.3"
requires: sandbox

spec: |
open_agent_spec: "1.6.1"
agent:
name: test
description: test
intelligence:
type: llm
engine: openai
model: gpt-4o
sandbox:
http:
allow_domains: [safe.example]
tasks:
first:
description: first
output: {type: object}
prompts: {system: "First.", user: "first"}
delegated:
description: blocked registry delegation
spec: oa://prime-vector/summariser
task: work
run:
description: run
depends_on: [first, delegated]
output: {type: object}
prompts: {system: "Run.", user: "run"}

mock_responses:
first: '{}'

invoke:
task: run
input: {}

expect_error:
code: SANDBOX_DOMAIN_VIOLATION
stage: sandbox
42 changes: 42 additions & 0 deletions spec/conformance/cases/sandbox/remote-delegation-preflight.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# Spec §11.3 — remote delegated specs MUST be checked before fetch/model execution.
description: "A blocked remote delegated-spec dependency fails during chain preflight"
spec_section: "11.3"
requires: sandbox

spec: |
open_agent_spec: "1.6.1"
agent:
name: test
description: test
intelligence:
type: llm
engine: openai
model: gpt-4o
sandbox:
http:
allow_domains: [safe.example]
tasks:
first:
description: first
output: {type: object}
prompts: {system: "First.", user: "first"}
delegated:
description: blocked remote delegation
spec: https://blocked.example/spec.yaml
task: work
run:
description: run
depends_on: [first, delegated]
output: {type: object}
prompts: {system: "Run.", user: "run"}

mock_responses:
first: '{}'

invoke:
task: run
input: {}

expect_error:
code: SANDBOX_DOMAIN_VIOLATION
stage: sandbox
Loading
Loading