Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,10 @@ jobs:
- name: Test (compatibility fixtures, round-trip byte identity, CLI, drift, action)
run: pnpm test

- name: Phase 10 deterministic release gates (A-T matrix, historical catalog, zero-touch closure)
if: matrix.os == 'ubuntu-latest' && matrix.node == 20
run: pnpm qualification:vnext-10-2:fast

- name: Large-repository performance suite (informational budgets)
# Heavy ~9,000-file fixtures: one representative combination is enough,
# and isolating it keeps the main test run's workers responsive.
Expand Down
20 changes: 20 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,26 @@

## Unreleased

### Production Qualification, Historical Fault Replay & Release Freeze — vNext.10.2 Phase 10

- Added a frozen production candidate identity bound to the exact commit,
tracked runtime bytes, public schema versions, and Claude/Codex bundle
digests. Release finalization detects candidate/runtime mutation, refuses
control-plane self-repair, preserves finalized failure evidence, and never
tags or publishes automatically.
- Added the mandatory A-T production gate manifest with
`PASS`/`FAIL`/`SKIPPED_NOT_ALLOWED`, candidate-bound evidence references,
deterministic `READY`/`NOT_READY`, and a `PRODUCTION_READY` marker that can
exist only when all twenty gates and zero-tolerance facts pass.
- Added a durable fourteen-entry StepRelay historical fault catalog and named
fast, fault-replay, Secondary, Research, cooldown, and soak qualification
groups. Representative Phase 10 deterministic gates now run explicitly in
CI in addition to the full test suite.
- Added `orchestrate qualify freeze` and one-shot `orchestrate qualify release`
surfaces, production reports/artifacts, Mission runtime pinning, credential-
shaped evidence refusal, and execution telemetry 1.1 fields for runtime
mutation, lost candidates, and logical duplicate dispatches.

### Telemetry, Token-Conservation Analytics & Operational Reporting — vNext.10.2 Phase 9

- Added a versioned Job/Mission execution telemetry report derived from
Expand Down
9 changes: 9 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,15 @@ investigation when enabled.
[Research Layer](docs/research/research-layer.md) ·
[Lifecycle Integration](docs/research/research-lifecycle.md)

**Production qualification and release freeze (vNext.10.2 Phase 10)** —
freeze one clean candidate, bind every A-T gate and all fourteen historical
StepRelay fault replays to its commit/runtime digest, then derive a strict
`READY` or `NOT_READY`. Missing real local-model, DeerFlow, soak, security,
closure, or reproducibility evidence cannot be waived into a pass, and the
`PRODUCTION_READY` marker is emitted only for a completely passing manifest.

[Production qualification](docs/orchestration/production-qualification.md)

**Spec workflow and approvals** — create Kiro-compatible specs offline
(`spec new`, four workflows, ten templates), analyze them
deterministically, and gate every stage behind an explicit human approval
Expand Down
15 changes: 15 additions & 0 deletions contracts/cli-commands.json
Original file line number Diff line number Diff line change
Expand Up @@ -575,6 +575,12 @@
"--json"
]
},
"freeze": {
"options": [
"--help",
"--json"
]
},
"preflight": {
"options": [
"--help",
Expand All @@ -584,6 +590,15 @@
"--target-name"
]
},
"release": {
"options": [
"--evidence",
"--help",
"--json",
"--markdown",
"--no-write"
]
},
"report": {
"options": [
"--help",
Expand Down
63 changes: 63 additions & 0 deletions contracts/orchestration-contract.json
Original file line number Diff line number Diff line change
Expand Up @@ -392,6 +392,22 @@
"MEDIUM",
"UNKNOWN"
],
"historicalFaultIds": [
"FAULT-001",
"FAULT-002",
"FAULT-003",
"FAULT-004",
"FAULT-005",
"FAULT-006",
"FAULT-007",
"FAULT-008",
"FAULT-009",
"FAULT-010",
"FAULT-011",
"FAULT-012",
"FAULT-013",
"FAULT-014"
],
"humanAttentionJobStatuses": [
"BLOCKED",
"NEEDS_AUTHORITY",
Expand Down Expand Up @@ -746,6 +762,53 @@
"MEDIUM",
"NONE"
],
"productionEvidenceKinds": [
"COMMIT",
"FIXTURE",
"LOG_ARTIFACT",
"QUALIFICATION_JSON",
"REPORT",
"TEST_RUN"
],
"productionGateIds": [
"brownfield-zero-touch",
"closure-completion",
"deerflow-failure-fallback",
"frontend-integration",
"full-repository-suite",
"greenfield-zero-touch",
"historical-fault-replay",
"public-contract-integrity",
"real-local-model",
"release-reproducibility",
"research-lifecycle",
"restart-resume",
"secondary-builder",
"secondary-repair",
"security-authority",
"strong-fallback",
"subscription-cooldown",
"telemetry-report",
"unattended-soak",
"workspace-bootstrap"
],
"productionGateResults": [
"FAIL",
"PASS",
"SKIPPED_NOT_ALLOWED"
],
"productionQualificationArtifacts": [
"PRODUCTION_READY.json",
"historical-fault-coverage.json",
"production-candidate.json",
"production-qualification-manifest.json",
"production-qualification-report.md",
"production-release-decision.json"
],
"productionReleaseDecisions": [
"NOT_READY",
"READY"
],
"profileFallbackLevels": [
"EXACT",
"HEURISTIC_PRIOR",
Expand Down
2 changes: 2 additions & 0 deletions contracts/schema-versions.json
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
"evaluationResult": "1.0.0",
"evidence": "1.0.0",
"executionPlan": "1.0.0",
"executionTelemetryReport": "1.1.0",
"extensionChecksums": "1.0.0",
"extensionManifest": "1.0.0",
"extensionProtocol": "1.0.0",
Expand All @@ -49,6 +50,7 @@
"orchestrationState": "1.0.0",
"overnightPreflight": "1.0.0",
"productBaseline": "1.0.0",
"productionQualification": "1.0.0",
"qualificationReport": "1.0.0",
"quotaSnapshot": "1.0.0",
"recoveryPlan": "1.0.0",
Expand Down
4 changes: 4 additions & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,10 @@ reference, grouped by area.
- [Dogfood & release qualification (vNext.9)](orchestration/dogfood-qualification.md) —
the scenario matrix, fail-closed preflight, scoped fault injection, state
invariant auditing, and the evidence-based release gate.
- [Production qualification and release freeze (vNext.10.2 Phase 10)](orchestration/production-qualification.md)
— frozen candidate identity, the mandatory A-T gate matrix, historical
StepRelay replay, real-resource/soak evidence, and deterministic
`READY`/`NOT_READY` release authority.
- [Orchestration configuration](orchestration/configuration.md) — every
policy knob, with defaults.

Expand Down
10 changes: 8 additions & 2 deletions docs/autonomy/execution-telemetry.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ specbridge report job <jobId> --json

The human view is concise by default. `--verbose` adds bounded WorkUnit
accounting and the cooldown timeline. The JSON view carries
`schemaVersion: "1.0.0"` and, unless `--no-persist` is used, is saved at
`schemaVersion: "1.1.0"` and, unless `--no-persist` is used, is saved at
`.specbridge/reports/job-<jobId>-telemetry.json`.

MCP clients can call the read-only `job_report` tool. It derives the same
Expand All @@ -38,6 +38,9 @@ exist. Unknown measurements are never reported as zero.
| `UsefulWorkDuringSubscriptionCooldown` | Distinct WorkUnits durably recorded as completed during a Strong subscription cooldown | Not a ratio | Includes productive Secondary and research progress while Strong-required work waits. Duplicate WorkUnit IDs within the same objective are counted once. |
| `ZeroTouchAfterSeal` | `true` only when `humanInterventionsAfterSeal` is zero | Not a ratio | Correct authority escalation is reported separately. A missing seal boundary is diagnosed by the underlying autonomy telemetry; it is never inferred from a successful outcome. |
| `CompletedWorkRedoCount` | Implementation attempts that began after the same WorkUnit had already been durably integrated | Not a ratio | Resource waits and replayed copies of the same durable attempt ID are excluded. Candidate reuse and restart recovery are reported separately. |
| `lostCandidates` | Candidates rebuilt after restart because the prior candidate could not be reused | Not a ratio | Must be zero for production qualification. |
| `duplicateDispatches` | Distinct attempt IDs with the same Objective, WorkUnit attempt, and builder kind | Not a ratio | Replayed copies of one durable attempt ID are deduplicated and do not count as execution. |
| `runtimeMutation` | `0` when candidate-bound runtime start/end digests match; `1` otherwise | Not a ratio | `null` outside a candidate-bound qualification because absence is not proof of stability. |

## Work accounting

Expand Down Expand Up @@ -73,7 +76,10 @@ cross-Job evidence cannot be mistaken for a Job-only total.

The report records Job and objective graph revisions, event and execution
ledger watermarks, seal references, the current non-secret strategy summary,
and whether the current runtime policy differs from the sealed binding.
whether the current runtime policy differs from the sealed binding, and—when
Phase 10 supplies a candidate-bound observation—the runtime start and end
digests. A Mission seal binding can persist the qualified version, commit,
digest, and qualification run ID without granting any additional authority.
Diagnostic text is bounded and redacted. Prompts, transcripts, raw provider
payloads, environment secrets, and credentials are never report fields.

Expand Down
4 changes: 4 additions & 0 deletions docs/orchestration/dogfood-qualification.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,10 @@ Everything here is **opt-in**. A workspace that never runs a qualification
command behaves exactly as it did before vNext.9: no file is created, no
policy changes, and no ordinary job is affected.

vNext.10.2 Phase 10 consumes this durable evidence through the stricter
[production qualification and release freeze](production-qualification.md).
The vNext.9 verdict is not itself a `PRODUCTION_READY` marker.

---

## The two modes
Expand Down
Loading
Loading