Skip to content

Colocate openprose-lint and add deterministic language gates - #145

Open
rawwerks wants to merge 4 commits into
mainfrom
codex/colocate-openprose-lint
Open

Colocate openprose-lint and add deterministic language gates#145
rawwerks wants to merge 4 commits into
mainfrom
codex/colocate-openprose-lint

Conversation

@rawwerks

@rawwerks rawwerks commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Adds openprose-lint as a Rust workspace crate under crates/openprose-lint.
  • Adds scripts/lint-prose.sh as the repository-owned deterministic gate for the current OpenProse language surface.
  • Wires the linter gate into pnpm lint and adds explicit advisory and packaging profiles.
  • Adds an OpenProse Lint GitHub Actions workflow that runs the linter gate on pull requests and main pushes.
  • Adds a curated packaged OpenProse spec bundle so cargo publish --dry-run -p openprose-lint proves the crate is self-contained without vendoring the full example/replay corpus.
  • Documents versioning, release, contributor, and package-readiness expectations for the colocated crate.

Use Case / Run Evidence

The practical invariant is: when a language surface lands on OpenProse main, the same branch's linter should know how to parse and validate it. Keeping the linter in this repository makes that invariant reviewable in the same PR that changes the language.

This change also gives maintainers a fast deterministic signal before reaching for heavier evaluation:

  • bash scripts/lint-prose.sh checks the current skill/spec snapshot, linter tests, conformance cases, and bundled examples.
  • bash scripts/lint-prose.sh advisory reports non-blocking language-surface discovery and optional drift checks.
  • bash scripts/lint-prose.sh package proves the crate package can be built and verified from the packaged source tree.
  • bash scripts/lint-prose.sh release-package is the clean-worktree package proof for actual release commits.

Design Boundary

The linter lives in crates/openprose-lint as a normal Cargo package because it is close to the language definition and should evolve with it. The root repository owns the gate script because maintainers and CI should not need to know Cargo internals to ask "does this OpenProse change still lint?"

The linter is not moved into the skill/spec docs themselves. The docs remain the human and agent-facing language contract; the Rust crate is the deterministic checker for that contract.

The linter does not decide GitHub Actions policy. scripts/lint-prose.sh is the single reusable entrypoint that can be run locally, from any CI runner, or from GitHub Actions. The workflow is deliberately thin: it installs toolchains and invokes pnpm lint:prose plus pnpm lint:prose:package.

The packaged spec bundle is intentionally curated. It includes the source files needed for build-time vocabulary extraction, source identity checks, linked language docs, and adapter manifest validation. The full example corpus, replay fixtures, generated TypeScript tests, and world-model binary fixtures remain in the parent repository and are linted from there; they are not part of the crate payload.

Examples

Run the default deterministic gate:

bash scripts/lint-prose.sh

Run non-blocking discovery output:

bash scripts/lint-prose.sh advisory

Verify package self-containment from a review branch:

bash scripts/lint-prose.sh package

Verify package self-containment from a clean release commit:

bash scripts/lint-prose.sh release-package

Testing

Validated locally:

pnpm lint:prose
bash scripts/lint-prose.sh release-package
git diff --check
ubs scripts/lint-prose.sh crates/openprose-lint/scripts/sync-spec-snapshot.sh crates/openprose-lint/src/spec_identity.rs
gitleaks git --no-banner --redact --log-opts=origin/main..HEAD --verbose

Additional targeted checks run during the branch:

pnpm test:skill
pnpm lint:prose:package
./scripts/bump-version.sh --check
PATH="$HOME/.local/opt/go/bin:$PATH" go run github.com/rhysd/actionlint/cmd/actionlint@v1.7.7 .github/workflows/openprose-lint.yml
bash -n scripts/lint-prose.sh
bash crates/openprose-lint/scripts/sync-spec-snapshot.sh --check
cargo test -p openprose-lint spec_source::tests::validation
cargo test -p openprose-lint discovery_reports_potential_undocumented_contract_sections
cargo test -p openprose-lint private_generation_suffix_is_not_a_public_alias
cargo run -p openprose-lint -- lint --profile strict packages/std/evals/prose-contributor.prose.md
cargo run -p openprose-lint -- capabilities packages/std/evals/prose-contributor.prose.md

bash scripts/lint-prose.sh release-package was checked from the clean local commit and passed without --allow-dirty. It packaged 109 files, 1.1MiB / 272.1KiB compressed, verified the packaged OpenProse spec bundle, and validated every packaged adapter manifest from the extracted package tree.

The package file list contains no spec-snapshot/.../examples/, replay/, world-models/, generated TypeScript example tests, generate.ts, or .bin fixtures.

Post-Merge Sequence

  1. Merge this PR and verify pnpm lint:prose plus pnpm lint:prose:package on main.
  2. Do not publish the crate as part of this PR.
  3. Publish openprose-lint only from a clean release commit, after maintainer approval, using the documented release-package gate.
  4. After first publish, add the OpenProse GitHub team as a crate owner.
  5. Once the colocated crate is the canonical source, update the standalone openprose/prose-lint repository to point contributors here, then archive that repository if maintainers agree no active workflow still depends on it.

Residual Risk / Follow-ups

  • pnpm lint now includes the linter gate. That is intentional for the invariant above, but maintainers should decide whether every warning should stay non-blocking or become blocking over time.
  • Example lint currently reports warnings in compatibility mode; this PR keeps them visible without making them blocking.
  • Actual crates.io publication is not part of this PR. It should happen only after explicit maintainer approval, from a clean release commit, with cargo credentials or trusted publishing configured, and with OpenProse crate ownership added after first publish.
  • The package profiles require jq in addition to Rust/Cargo.
  • Because the workflow is new in this PR, GitHub's first live proof is the pull-request run itself. If Actions does not register the workflow immediately, a branch synchronize event after PR creation should be enough to re-check it.

@rawwerks

rawwerks commented Jun 26, 2026

Copy link
Copy Markdown
Contributor Author

Follow-up correction after package-shape review:

I narrowed crates/openprose-lint/spec-snapshot/openprose from a broad copy of skills/open-prose/ to a curated packaged spec bundle. The package still includes the files needed for build-time vocabulary extraction, source identity checks, linked core language docs, and adapter manifest validation, but it no longer vendors the full example corpus, replay fixtures, generated TypeScript tests, or world-model .bin artifacts.

I also tightened scripts/lint-prose.sh release-package / package so the extracted packaged crate validates specs verify --spec openprose and every packaged adapter manifest from the packaged source tree. That caught guidance/system-prompt.md, which is intentionally included because the adapter manifests reference it.

New local proof from the clean commit:

pnpm lint:prose
bash scripts/lint-prose.sh release-package

The release package dry-run now packages 109 files, 1.1MiB / 272.1KiB compressed, and the package file list contains no spec-snapshot/.../examples/, replay/, world-models/, generated TypeScript example tests, generate.ts, or .bin fixtures.

@irl-dan i think ready for your review

@rawwerks

Copy link
Copy Markdown
Contributor Author

Why the crate contains a spec snapshot

A visible part of this PR is a curated OpenProse spec bundle under
crates/openprose-lint/spec-snapshot/openprose. It is not a second copy of the
implementation.

openprose-lint derives part of its behavior and source-identity checks from
the canonical OpenProse language docs in skills/open-prose, including
SKILL.md, Contract Markdown, Prose VM, Forme, ProseScript, compiler, state,
Reactor, dependency, and runtime-contract docs. In this monorepo the crate can
read those files from the parent checkout. After cargo package, cargo publish, or cargo install, the crate is extracted by itself and files outside
crates/openprose-lint are not available. Without an in-crate bundle, packaged
builds and spec verification would either fail or fall back to incomplete
hardcoded behavior.

The snapshot is the better tradeoff because skills/open-prose remains the
source of truth while the Rust package stays self-contained for linter builds,
spec-identity verification, and adapter manifest validation. The alternatives
are worse: network fetches during build, requiring a parent checkout users do
not have, duplicating the language surface in Rust constants, or shipping a
crate that behaves differently from the repo gate.

Maintenance is mechanical:

  • The source docs under skills/open-prose remain authoritative.
  • crates/openprose-lint/scripts/sync-spec-snapshot.sh --sync refreshes the
    curated bundle from an explicit allowlist.
  • crates/openprose-lint/scripts/sync-spec-snapshot.sh --check fails if the
    bundle drifts or contains files outside that allowlist.
  • scripts/lint-prose.sh runs that check in the deterministic repo gate.
  • scripts/lint-prose.sh package and release-package prove the extracted
    Cargo package can verify the bundled spec and adapter manifests without
    reaching back into the monorepo.

The snapshot is intentionally not the full repo or full example corpus. It
excludes examples, replay fixtures, generated TypeScript tests, world-model
binaries, and other large/non-package artifacts. Those stay in the parent repo
and are linted there.

@rawwerks

Copy link
Copy Markdown
Contributor Author

I tightened the packaging/review boundary in two ways so the generated snapshot is still available to the Rust package, but is less prominent as review surface.

First, openprose-lint has an explicit [package] include allowlist in Cargo.toml. The snapshot is still generated by scripts/sync-spec-snapshot.sh, but the crate payload is declared through Cargo instead of being implicit.

Second, .gitattributes marks the snapshot path as generated:

crates/openprose-lint/spec-snapshot/** linguist-generated

That should keep those files hidden by default in GitHub diffs while still including them for cargo package / cargo install.

So the intended review path is: review the source docs, sync script, Cargo include allowlist, and package verification gate; treat spec-snapshot/** as generated output. I also re-checked this locally with sync-spec-snapshot.sh --check, cargo package -p openprose-lint --allow-dirty --list, git check-attr, and git diff --check.

This does not make the snapshot disappear, but it makes the boundary explicit and gives future changes guardrails as long as we keep using the sync/check/package steps intentionally.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant