Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,12 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and
build artifact. It never modifies repository code; its optional agent-interpretation
step is disabled by default and defines no credentials. Minimal per-job permissions,
`persist-credentials: false`, and full-SHA-pinned actions throughout.
- The shared validator is now consumed as the deterministic `engineering-loop-schemas v0.1.2`
bundle pinned to `0459d61b7b1d4e7b46709e6d3895770553e6fab0`. Its `manifest.json` records
source provenance, file sizes, SHA-256 hashes, and the declared package-import adaptation.
The mandatory `loop-schema-vendor` quality check validates the bundle offline, and regression
tests prove that manual tampering is rejected. The earlier `75a63eef...` integration described
below was an intermediate pre-release state and is superseded by this published bundle.

### Fixed

Expand Down
66 changes: 66 additions & 0 deletions VALIDATION.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,71 @@
# Validation record

## Current validation — Phase 0-1

Validated on 2026-07-18 for the report-only Evidence-Gated Engineering Loop foundation.

### Identity and provenance

- Published repository commit:
`59b29429c4b50984e2ceffd9f411ef2028a72e8b`.
- Harness generator version: `1.2.1` (`bootstrap.py` `HARNESS_VERSION`).
- Shared schemas: `v0.1.2` at
`0459d61b7b1d4e7b46709e6d3895770553e6fab0`.
- Final integration: pull request
[#8](https://github.com/brunovicco/codex-python-engineering-harness/pull/8).
- The pull-request quality run validated source head
`ed0f5bd6827bba8cfef7948bb339b694dbfb62d5` before GitHub recreated the commit through squash
merge:
<https://github.com/brunovicco/codex-python-engineering-harness/actions/runs/29660517744>.

### Results

- Repository quality gate: passed.
- Regression tests: 70 passed.
- Ruff lint and format check: passed.
- Strict Mypy: passed.
- Bandit medium/high findings: none.
- `pip-audit`: no known vulnerabilities.
- `loop-schema-vendor`: passed for `v0.1.2`.
- Positive bundle-integrity test: passed.
- Manual-tampering detection test: passed.
- Legacy provider-reference regression test: passed.
- Harness self-evaluation: `Overall: PASS`.
- Complete quality gates passed for all six profile/governance combinations:
- `service-none`;
- `service-agentic`;
- `library-none`;
- `library-agentic`;
- `workspace-none`;
- `workspace-agentic`.

### Scope and limitations

This validation covers Phase 0-1 only. The repository can validate contracts, execute existing
quality gates, verify vendored-schema integrity, render temporary profiles, and produce reports.
It does not provide a loop runner, state machine, evaluator runtime, autonomous candidate creation,
candidate promotion, merge, or deployment.

The builder's report remains non-authoritative. A quality gate is the technical authority, and
human review remains required before any promotion decision.

The self-evaluation workflow still uses the previously pinned `setup-uv` action while the main
quality workflow uses the newer approved pin. Aligning those pins is CI maintenance and should be
performed in a separate change from this documentation record.

### Reproduce the Phase 0-1 validation

```bash
uv lock --check
uv sync --frozen --all-groups
uv run python scripts/quality_gate.py
uv run python scripts/loop_self_evaluation.py \
--output-dir build/loop-self-evaluation
git diff --check
```

## Previous validation — 2026-07-16

Validated on 2026-07-16 (America/Sao_Paulo) with Python 3.12 for the harness and Python 3.13 for
generated profiles.

Expand Down
27 changes: 17 additions & 10 deletions docs/LOOPS.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,10 +51,13 @@ Every loop run belongs to one of three levels of scrutiny, mirrored from
- A builder never certifies its own result. Only a mechanically-derived
`verdict` can.
- A hard gate is default-FAIL and must reduce to a command with an exit
code. The set of hard gates a contract may reference
(`acceptance.hard_gates`) is exactly the named checks this harness's own
code. The contract-addressable hard gates
(`acceptance.hard_gates`) are exactly the named checks this harness's own
`quality_gate.py` implements: `lock`, `lint`, `format`, `typing`,
`tests`, `security`, `dependencies`, `architecture`, `mcp`, `governance`.
Separate mandatory infrastructure checks, including `loop-schema-vendor` and
`loop-contracts`, always protect provenance, integrity, and contract validation; they are not
arbitrary commands selectable by a builder.
- Evidence is bound to exact commits (`baseline_sha`, `candidate_sha`) and
a hashed environment (`uv_lock_sha256`), so a verdict can always be
traced back to exactly what ran against exactly what code.
Expand All @@ -81,14 +84,18 @@ Every completed run resolves to exactly one final state

## Vendoring

`scripts/_vendor_loop_schemas/` (and its `template/scripts/` copy shipped
to generated projects) is a verbatim vendored copy of
`engineering-loop-schemas`' `src/loop_schemas/` at a pinned commit, recorded
in a header comment in each vendored file. Re-vendor from the source
repository rather than hand-editing; the one intentional deviation (the
package directory is named `_vendor_loop_schemas`, not `loop_schemas`, so
it does not collide with the `scripts/loop_*` denylist above) is documented
in that same header.
`template/scripts/_vendor_loop_schemas/` is a deterministic bundle rendered from
`engineering-loop-schemas v0.1.2` at the full commit
`0459d61b7b1d4e7b46709e6d3895770553e6fab0`. Its `manifest.json` records the source repository,
version, commit, file sizes, SHA-256 hashes, and the declared import adaptation.

The bundle is not a byte-for-byte copy. During rendering, the package import in
`validate_contract.py` changes from `loop_schemas` to `_vendor_loop_schemas`; this keeps the
vendored package isolated and avoids collision with the protected `scripts/loop_*` namespace. The
adaptation is explicit in the manifest and covered by deterministic-rendering, integrity, and
tampering tests. `loop-schema-vendor` verifies the bundle offline on every generated-project
quality run. Fix the canonical schemas repository and render a new version instead of editing the
bundle manually.

`validate_contract.py` is stdlib-only. Reading a YAML contract requires
PyYAML to be importable in the environment `scripts/quality_gate.py` runs
Expand Down