Skip to content

fix: validate metadata-only release readiness evidence - #486

Merged
leehack merged 3 commits into
mainfrom
fix/metadata-release-readiness
Sep 7, 2026
Merged

fix: validate metadata-only release readiness evidence#486
leehack merged 3 commits into
mainfrom
fix/metadata-release-readiness

Conversation

@leehack

@leehack leehack commented Sep 7, 2026

Copy link
Copy Markdown
Owner

Summary

Closes #485.

  • Add an explicit, evaluator-verified core-patch release metadata route without fabricating changed production tests or classifying artifact-consumer releases as standard risk.
  • Require literal exact-base/head Git blobs and a complete fixed-path inventory: only the next core patch scalar, matching generated local chat lock version, current release notes and four maintained install docs. Companion versions/pins are already prepared and must not change here.
  • Preserve independent exact-head audit, zero P1/zero unresolved-thread gates, strict schema/row commands, and external authenticated auditor/App/environment/ruleset boundaries.

Scope and safety

The metadata route requires release-metadata-verification, its exact strict-release-verifier/test command, PASS evidence and exactly the unchanged test/unit/tooling/verify_release_docs_companion_pins_test.dart. The verifier and test must be identical regular Git blobs at base/head. The evaluator does not execute candidate scripts or authenticate a claimed test run; independent review must inspect actual command evidence.

Any hook/runtime/pin/binding/Swift/workflow/policy change, broader companion release, renamed/added/deleted/nonregular path, dependency/SDK/source/hash/path/inventory lock change, stale snippet, historical changelog rewrite or executable MDX activation cannot use this route. Documentation prose remains independently reviewed. Normal high-risk changes retain the changed-production-test requirement.

This fixes a pre-existing policy mismatch exposed by release PR #482, not a runtime regression caused by that release or Apple guard #484. Release #482 stays draft and is not modified by this corrective PR.

Validation

  • PASS: canonical Flutter3.47.1/Dart3.13.1 preparation, whole format (590 files unchanged) and analysis.
  • PASS: final serial full VM 2,046 tests /77 fixture-dependent skips.
  • PASS: focused evaluator 102 tests; combined readiness/classifier/matrix/release-verifier contracts 151/151, including real temporary Git repository exact-blob and stale-head tests.
  • PASS: Node20 production docs build and strict links; git diff check.
  • PASS: live read-only replay of release chore: prepare 0.8.23 release #482 exact head 16eaa80fcdb7578599d8757056cfb18536a642cb / base ccc2d35633e3ca6347a697f19fde4944e0a8dbd0 reaches unverifiedPrerequisites instead of missingTestPath without changing its diff or inventing a changed test.
  • Adversarial coverage includes code/pin/workflow/policy mixtures, pubspec/lock mutation, file modes, stale/self audit, P1/threads, verifier/test tampering, wrong/malformed evidence, historical docs, fenced/inline code-to-MDX activation, Docusaurus executable fences, and unavailable external prerequisites.
  • Real model/device smoke: N/A to developer-only evidence tooling. No public inference, pins, artifacts or runtime behavior changes.

High-risk regression review

  • Classification: high-risk / regressionPolicy.
  • Implementation task: current_release_prep, isolated /private/tmp/llamadart-readiness-485.
  • Exact head: a8bb6ff318d2ef028b81f549cd1aefe7929abf07; base ccc2d35633e3ca6347a697f19fde4944e0a8dbd0.
  • Independent Astra final exact-head review: PASS; independently reran 102 evaluator tests. Fence activation, JSX template-literal and script-body review findings were fixed with durable adversarial regressions before readiness.
  • Current-head hosted checks: 12/12 SUCCESS, including replacement CI run 34159032954 and the read-only advisory. Linux coverage and aggregate passed. Review threads: 0 total /0 unresolved. No automated approving review is claimed.
  • Known PR-caused P1 regressions: zero after independent review.
  • Separate pre-existing tooling issue Exclude generated website output from maintained writer source scan #487 tracks the maintained-source scanner reading generated website build files. One local full-VM run overlapped docs generation and hit an ENOENT in that unchanged scanner; the authoritative serialized rerun passed 2,046 tests. This PR does not weaken or modify that scanner.

Explicit boundary

Successful local high-risk evaluation still returns only unverifiedPrerequisites (exit2), isReady=false. No payload, flag, credential or environment variable can produce operational-ready output. Protected external enforcement remains separately governed and unconfigured by design; this PR neither enables nor weakens it.

No mark-ready, merge, publication, tag, dispatch, settings or runtime-pin action is included.

@leehack
leehack marked this pull request as ready for review September 7, 2026 21:26
@leehack
leehack merged commit dee010e into main Sep 7, 2026
12 checks passed
@leehack
leehack deleted the fix/metadata-release-readiness branch September 7, 2026 21:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support honest metadata-only release evidence in the high-risk evaluator

1 participant