Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions benchmarks/preflight/PREFLIGHT-EXECUTION-READINESS-v0.1.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# OIC-Bench Preflight Execution Readiness v0.1

Status: preparation only. Semantic OIC-Bench has not been run. This document
does not authorize source interpretation, admission, OCE generation, Rego
generation, or runtime semantic evaluation.

| Readiness area | Current state | Required before execution | Expected evidence/failure behavior |
|---|---|---|---|
| Frozen governing-source inventory | INCOMPLETE | Owner-selected minimum source roles; complete manifest and freeze | Missing source evidence blocks ingestion |
| Manifest completeness | INCOMPLETE / exit 3 | Every selected source has rights, provenance, hash, dates and authority status | Exit 3 remains missing evidence, never pass |
| Annotation/gold prerequisites | NOT STARTED | Schema, annotator roles, adjudication procedure, disagreements and immutable gold version | No fake labels or model-generated gold |
| Metric implementation | PROPOSED / NOT MEASURED | Freeze metric definitions and deterministic calculators | Invalid/undefined denominators fail closed |
| Benchmark lineage | PARTIAL | Bind source, annotation, compiler, schema, model, admission and test versions | Every raw result traces to exact inputs |
| Baseline harness | NOT STARTED | Direct LLM-to-Rego and other selected baselines use same inputs/model where required | No comparative claim from missing baseline |
| Environment capture | INFRASTRUCTURE READY | Lock files, tool versions, offline/network declaration, seeds and hardware/runtime metadata | Environment drift is reported |
| Raw-result destinations | NOT ESTABLISHED | Versioned machine-readable raw outputs, logs, hashes and failure inventory | Never publish only summaries |
| Reproducibility commands | PARTIAL | One clean command per stage plus expected exit codes | Non-zero/INCOMPLETE states retained |
| Source-version/change-impact cases | NOT STARTED | At least three authorized version/amendment pairs for preflight | Missing affected-control evidence is visible |
| Runtime transaction cases | NOT STARTED | At least 20 adjudicated positive, negative, boundary and unknown cases | `CANNOT`/escalation can be correct |
| Ambiguity/conflict cases | NOT STARTED | At least six gold blocking ambiguity/conflict/missing-authority cases | False resolution is a failure |
| Malicious/adversarial sources | NOT STARTED | At least one malicious instruction plus negation, threshold, time, actor and exception mutations | Source instructions cannot alter authority |
| Claim ceiling | ESTABLISHED | Preserve preflight ≠ full OIC-Bench | No generalization, enterprise proof, superiority or maturity claim |

## Immediate post-gate sequence

1. Verify the selected frozen-source manifest; stop on `INCOMPLETE`.
2. Freeze annotation protocol and gold roles before semantic output is scored.
3. Capture the exact environment and all dependency/model versions.
4. Run semantic stages only under the separately authorized bounded profile.
5. Store raw outputs, refusals, errors, lineage and hashes.
6. Run preregistered preflight metrics and named baselines without changing
thresholds after observing results.
7. Publish failures and limitations with any bounded experimental result.

Preflight evidence is not full OIC-Bench v0.1 and cannot establish statistical
generalization, enterprise suitability, comparative superiority, or product
maturity.
66 changes: 66 additions & 0 deletions docs/gates/OIC-PREFLIGHT-PROVENANCE-GAP-REPORT-v0.1.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# OIC Preflight Provenance Gap Report v0.1

Status: mechanical audit at OIC commit
`d06917fa6877277d7118b49e80d6a69446f50712`. No rights, authority, effective
date, or source meaning is inferred.

## Governing requirement

`benchmarks/preflight/README.md` requires every source used by the preflight to
appear in `SOURCE_MANIFEST.csv` with rights, provenance, immutable hash,
effective dates, and declared authority status. The proposed Canada working set
is defined by `benchmarks/preflight/canada/WORKING-SET-SCOPE-v0.1.md` and
`SOURCE-REGISTRY-PROPOSED-v0.1.json`; proposals are not frozen sources.

Evidence status vocabulary: `ESTABLISHED`, `PARTIAL`, `OPEN`,
`OWNER/DOMAIN ATTESTATION REQUIRED`, and `NOT REQUIRED FOR CURRENT PREFLIGHT`.

## Source-by-source provenance gap matrix

The “field state” column follows this order: stable ID; title; issuer; type;
official origin; reuse basis; acquisition time; SHA-256; effective-from;
effective-until; supersession/amendment; confidentiality; verification
receipt/command; benchmark authority status; frozen bytes.

| Source | Field state | Current evidence and exact artifact/blob | Overall status | Missing evidence | Required authority | Smallest truthful next action | Gate effect |
|---|---|---|---|---|---|---|---|
| CA-1 | E,E,E,E,E,P,O,O,O,O,P,P,O,O,O | Proposed registry and scope only: `SOURCE-REGISTRY-PROPOSED-v0.1.json` blob `35e23be5…`; `WORKING-SET-SCOPE-v0.1.md` blob `cf2ce8a9…` | PARTIAL | Rights clearance, authorized acquisition, frozen bytes/hash/receipt, effective dates, confidentiality and authority declaration | Owner plus qualified rights/domain reviewer | Resolve rights; if permitted, authorize deterministic acquisition and freeze | BLOCKS SEMANTIC CODE START |
| CA-2 | E,E,E,E,E,P,O,O,O,O,P,P,O,O,O | Same proposed registry/scope; appendix is version-related to CA-1 | PARTIAL | Same gaps as CA-1 plus final table/node anchors | Owner plus qualified rights/domain reviewer | Resolve rights and freeze selected appendix nodes | BLOCKS SEMANTIC CODE START |
| CA-3 | E,E,E,E,E,E,E,E,O,O,P,E,E,O,E | Manifest blob `718a9df6…`; frozen XML blob `9d89e621…`; receipt blob `52f8585f…`; freeze record blob `dc2a89d…`; rights clearance blob `dace02fd…`; SHA-256 `6e89ad25…` | PARTIAL | Effective-from/effective-until state and declared benchmark authority status are blank; current verifier still reports corpus evidence `INCOMPLETE` | Owner/domain authority for benchmark authority; domain reviewer for effective-date treatment | Record bounded authority/effective-date disposition without inventing dates, then make the current verifier resolve the accepted frozen receipt | BLOCKS SEMANTIC CODE START |
| CA-4 | E,E,E,E,E,P,O,O,O,O,P,P,O,O,O | Proposed registry/scope only (`35e23be5…`, `cf2ce8a9…`) | PARTIAL | Rights, freeze, effective date, authority and receipt fields | Owner plus qualified rights/domain reviewer | Resolve rights and freeze selected provisions if permitted | BLOCKS SEMANTIC CODE START |
| CA-5-APPROVALS | E,E,E,E,E,O,O,O,O,O,P,P,O,O,O | Proposed registry; rights-clearance record says publisher robots policy refuses project retrieval and reuse evidence is unresolved (`RIGHTS-CLEARANCE-v0.1.json`, blob `dace02fd…`) | OWNER/DOMAIN ATTESTATION REQUIRED | Lawful acquisition/reuse path, frozen bytes, dates, receipt, authority | Owner plus qualified licensing reviewer; publisher if permission is sought | Replace with a rights-clear source or obtain permission and record it | BLOCKS SEMANTIC CODE START |
| CA-5-DELEGATION | E,E,E,E,E,O,O,O,O,O,P,P,O,O,O | Same CanadaBuys rights evidence as CA-5-APPROVALS | OWNER/DOMAIN ATTESTATION REQUIRED | Same; public guidance also does not prove an individual delegation | Owner, licensing reviewer, domain authority | Select a lawful source and define the delegation-evidence ceiling | BLOCKS SEMANTIC CODE START |
| CA-5-SIGNING | E,E,E,E,E,O,O,O,O,O,P,P,O,O,O | Same CanadaBuys rights evidence | OWNER/DOMAIN ATTESTATION REQUIRED | Rights/acquisition/freeze/date/authority evidence | Owner, licensing reviewer, domain authority | Replace or obtain permission; then freeze | BLOCKS SEMANTIC CODE START |
| CA-5-LIMITS | E,E,E,E,E,O,O,O,O,O,P,P,O,O,O | Same CanadaBuys rights evidence | OWNER/DOMAIN ATTESTATION REQUIRED | Rights/acquisition/freeze/date/authority evidence | Owner, licensing reviewer, domain authority | Replace or obtain permission; then freeze | BLOCKS SEMANTIC CODE START |
| CA-6-ARCHIVE | E,E,E,E,E,O,O,O,P,P,E,P,O,O,O | Proposed archived-source metadata and rights refusal evidence | OWNER/DOMAIN ATTESTATION REQUIRED | Lawful frozen copy, hash/receipt, exact historical period, authority classification | Owner, licensing reviewer, domain authority | Replace with a lawfully reusable stale/superseded source or obtain permission | BLOCKS SEMANTIC CODE START |
| CA-6-CH6 | E,E,E,E,E,O,O,O,P,P,E,P,O,O,O | Seven proposed stable section numbers in scope; no acquired bytes | OWNER/DOMAIN ATTESTATION REQUIRED | Same as CA-6-ARCHIVE plus frozen node boundaries | Owner, licensing reviewer, domain authority | Resolve lawful source and freeze exact nodes | BLOCKS SEMANTIC CODE START |
| CA-6-GLOSSARY | E,E,E,E,E,O,O,O,O,O,P,P,O,O,O | Scope assigns zero entries because no direct selected-node reference was evidenced | NOT REQUIRED FOR CURRENT PREFLIGHT | None while zero-entry exclusion remains owner-accepted | Owner only if scope changes | Retain exclusion | DOES NOT CURRENTLY BLOCK |
| CA-7 | P,P,P,P,P,O,O,O,O,O,O,O,O,O,O | Explicitly excluded by current 55-page scope | NOT REQUIRED FOR CURRENT PREFLIGHT | None unless owner expands scope | Owner only if scope changes | Retain exclusion | DOES NOT CURRENTLY BLOCK |

Legend: `E` established, `P` partial, `O` open.

## Result

No source currently satisfies every preflight provenance requirement. CA-3 is
the only source with frozen bytes, a cryptographic digest, a tracked receipt,
and documented reuse basis. It remains incomplete for the gate because the
manifest lacks effective-date and declared authority dispositions and the
current manifest verifier reports `INCOMPLETE`.

The smallest truthful source blocker set is:

1. close the bounded CA-3 effective-date/authority/verification disposition;
2. owner-select a minimum source set that covers the required source roles;
3. replace or clear sources whose acquisition/reuse is blocked;
4. freeze every selected source with bytes, hashes, receipts, dates where
supportable, confidentiality, and declared benchmark authority status.

## Owner source-strategy choice

The governing design permits a bounded public or synthetic subset. The owner
may continue real Canada clearance, or authorize CA-3 as the real anchor plus
explicitly synthetic procurement sources for remaining roles. Synthetic
companions must name a synthetic test institution, declare their synthetic
status, carry deterministic generated provenance and hashes, create no
real-world authority, and remain limited to benchmark/test semantics. This
audit does not authorize or create them.
110 changes: 110 additions & 0 deletions docs/gates/OIC-SEMANTIC-CODE-START-GATE-CLOSURE-v0.1.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
# OIC Semantic Code-Start Gate Closure v0.1

Original audit base: `d06917fa6877277d7118b49e80d6a69446f50712`.
The current comparison base is recorded in draft PR #35 after main
reconciliation. `STATUS.md` remains authoritative and unchanged.

## Corrected closure matrix

| Gate | Requirement | Current evidence/status | Smallest next action | Effect |
|---|---|---|---|---|
| OIC-CS-001 | Selected bounded source strategy and complete provenance | CA-3 frozen/rights-cleared but incomplete; other real sources proposed or blocked — OPEN | Owner selects Path A/B; complete selected records | BLOCKS SEMANTIC CODE START |
| OIC-CS-002 | Bounded ZTL profile admitted | Signed pin, 13/3 conformance, corrected mapping, 28 fields, WP-3, SC-WA and triggers — TECHNICALLY ESTABLISHED / OWNER ADMISSION PENDING | Correct stale metadata in admission package; owner accepts/rejects profile and decides Tier-1 timing | BLOCKS SEMANTIC CODE START |
| OIC-CS-003 | Minimum VEIP handoff admitted | Technical non-executable option identified; external heads unadmitted — PARTIAL | Owner accepts minimum boundary or requires full lifecycle | BLOCKS SEMANTIC CODE START |
| OIC-PF-001 | Result-bearing semantic preflight | Gold/runtime/change/adversarial cases and execution-grade lifecycle absent | Complete only after code-start authorization | BLOCKS PREFLIGHT EXECUTION |
| OIC-ER-001 | Reproducible experimental release | Tier-1 disposition and VEIP conformance/security incomplete | Independent/external evidence later | BLOCKS EXPERIMENTAL RELEASE |
| OIC-RF-001 | Full benchmark/maturity | Not run/not established | Defer | BLOCKS RELEASE FREEZE |

## Minimum true blocker set

1. **Source-set owner decision plus provenance completion.**
2. **ZTL profile owner admission**, not new ZTL semantic design; includes stale
metadata correction and independent-reproduction timing.
3. **Minimum VEIP boundary owner admission**, not completion of the full VEIP
lifecycle product.

No additional code-start blocker is asserted without an exact governing
requirement.

## Corpus alternatives

**Path A — real Canada clearance:** finish rights, bytes, hashes, receipts,
dates, confidentiality and benchmark authority for the selected real set.

**Path B — mixed public/synthetic code-start set:** subject to owner
authorization, retain CA-3 as the real anchor and create deterministic synthetic
companions for the remaining roles. Each must name a synthetic test institution,
declare synthetic status, carry generated provenance and fixed hashes, create no
real-world authority and support benchmark/test semantics only.

Path B may accelerate code start. It does not establish real-world corpus
completeness, enterprise generalization, superiority or legal validity. Neither
path is executed here.

## Horizon separation

| Horizon | Required state |
|---|---|
| Code start | Selected source path provenance-complete for bounded scope; exact ZTL profile admitted; minimum fail-closed/non-executable VEIP handoff admitted |
| Result-bearing preflight execution | Authorized semantic implementation, gold/metric lineage, runtime cases and execution-grade VEIP lifecycle for the tested path |
| Experimental release | Frozen reproducible preflight, raw results, Tier-1 disposition, VEIP conformance/security and bounded claims |
| Release freeze | Full OIC-Bench/held-out, independent review, usability, security, reliability and maturity gates |

## OWNER DECISIONS REQUIRED TO OPEN BOUNDED SEMANTIC CODE START

### OIC-CS-OD-001 — Source strategy

- **Question:** Continue real Canada clearance or authorize CA-3 plus bounded
synthetic companions?
- **Option A:** Continue Path A.
- **Option B:** Authorize Path B with its strict claim ceiling.
- **Recommended conservative default:** A unless speed justifies B's limits.
- **Evidence:** provenance report; governing design permits public or synthetic.
- **Authorizes:** provenance work for the chosen path.
- **Still prohibits:** semantic execution and real-world authority/result claims.

### OIC-CS-OD-002 — ZTL profile admission

- **Question:** Accept profile `ztl-v0.1` version `0.1.0`, signed tag
`veraxis-ztl-input-v0.2-signed`, commit `56e1ff05…`, fixture index
`ffadd653…` for bounded code start?
- **Option A:** Accept after correcting stale provenance metadata.
- **Option B:** Keep blocked and name precise additional evidence.
- **Recommended conservative default:** A only while provisional boundaries,
hazards and claim ceilings remain explicit.
- **Evidence:** merged PR #18/#16 and current accepted contracts/review.
- **Authorizes:** bounded semantic code targeting the admitted profile.
- **Still prohibits:** independence/maturity/authority claims and operational
authorization by ZTL.

### OIC-CS-OD-003 — ZTL independent-reproduction timing

- **Question:** Require Tier-1 before code start or defer it?
- **Option A:** Require before code start.
- **Option B:** Defer to experimental release/release freeze.
- **Recommended conservative default:** B, while all independence claims remain
prohibited.
- **Evidence:** dossier §13 and v0.2 producer conformance.
- **Authorizes:** timing only.
- **Still prohibits:** calling producer evidence independent.

### OIC-CS-OD-004 — Minimum VEIP boundary

- **Question:** Accept a non-executable review-only handoff for compiler code
start, or require the full lifecycle first?
- **Option A:** Accept: ActionProposal precedes OIC; OIC consumes exact context
and emits RuntimeDecision evidence; VEIP does not reinterpret ZTL; neither
creates the other's authority; missing integration is fail-closed.
- **Option B:** Require the full VEIP lifecycle first.
- **Recommended conservative default:** A for bounded compiler work only, with
result-bearing execution separately blocked.
- **Evidence:** current warrant contract and ZTL-VEIP preflight; external heads
remain unadmitted.
- **Authorizes:** bounded compiler contracts without a VEIP execution adapter.
- **Still prohibits:** execution, reliance, lifecycle adapters, operational
publication and VEIP maturity claims.

## Gate answer

**SEMANTIC CODE-START GATE: BLOCKED — OWNER DECISIONS NOW ISOLATED**

Loading