Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -470,6 +470,10 @@ jobs:
run: cargo run -p awa --example etl_pipeline
env:
DATABASE_URL: postgres://postgres:postgres@localhost:5432/awa_test
- name: Run Rust quickstart example
run: cargo run -p awa --example quickstart
env:
DATABASE_URL: postgres://postgres:postgres@localhost:5432/awa_test

# ─── Chaos smoke test ─────────────────────────────────────────
# Lightweight version of the nightly chaos suite — catches Python
Expand Down
142 changes: 142 additions & 0 deletions .github/workflows/docs.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,142 @@
name: Docs

on:
push:
branches: [main]
paths:
- .github/workflows/docs.yml
- awa/examples/quickstart.rs
- awa-python/examples/quickstart.py
- docs/**
- mkdocs.yml
- requirements-docs.txt
- scripts/build-agent-docs.py
- scripts/check-docs.sh
pull_request:
branches: [main]
types: [opened, reopened, synchronize, closed]
paths:
- .github/workflows/docs.yml
- awa/examples/quickstart.rs
- awa-python/examples/quickstart.py
- docs/**
- mkdocs.yml
- requirements-docs.txt
- scripts/build-agent-docs.py
- scripts/check-docs.sh
workflow_dispatch:

concurrency:
# Serialize each preview's updates and cleanup without letting unrelated PRs
# evict one another from GitHub's single pending slot. Production deploys
# remain serialized separately; manual runs are intentionally independent.
group: >-
awa-docs-${{
github.event_name == 'pull_request'
&& format('pr-{0}', github.event.pull_request.number)
|| github.event_name == 'push'
&& 'production'
|| format('manual-{0}', github.run_id)
}}
cancel-in-progress: false
Comment thread
hardbyte marked this conversation as resolved.

permissions:
contents: read

jobs:
build:
if: github.event_name != 'pull_request' || github.event.action != 'closed'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6

- uses: actions/setup-python@v6
with:
python-version: "3.12"
cache: pip
cache-dependency-path: requirements-docs.txt

- uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@v2

- name: Install documentation dependencies
run: python -m pip install --requirement requirements-docs.txt

- name: Build docs and check canonical examples
run: scripts/check-docs.sh

- name: Prepare Pages artifact
run: touch site/.nojekyll

- uses: actions/upload-artifact@v7
with:
name: awa-docs-${{ github.run_id }}
path: site
include-hidden-files: true
if-no-files-found: error

preview:
if: >-
github.event_name == 'pull_request' &&
github.event.action != 'closed' &&
github.event.pull_request.head.repo.full_name == github.repository
needs: build
runs-on: ubuntu-latest
permissions:
contents: write
pull-requests: write
steps:
- uses: actions/checkout@v6

- uses: actions/download-artifact@v8
with:
name: awa-docs-${{ github.run_id }}
path: site

- name: Publish pull request preview
uses: rossjrw/pr-preview-action@ffa7509e91a3ec8dfc2e5536c4d5c1acdf7a6de9 # v1.8.1
with:
source-dir: site
preview-branch: gh-pages
umbrella-dir: pr-preview

remove-preview:
if: >-
github.event_name == 'pull_request' &&
github.event.action == 'closed' &&
github.event.pull_request.head.repo.full_name == github.repository
runs-on: ubuntu-latest
permissions:
contents: write
pull-requests: write
steps:
- uses: actions/checkout@v6

- name: Remove pull request preview
uses: rossjrw/pr-preview-action@ffa7509e91a3ec8dfc2e5536c4d5c1acdf7a6de9 # v1.8.1
with:
action: remove
preview-branch: gh-pages
umbrella-dir: pr-preview

deploy:
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
needs: build
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- uses: actions/checkout@v6

- uses: actions/download-artifact@v8
with:
name: awa-docs-${{ github.run_id }}
path: site

- name: Publish production site
uses: JamesIves/github-pages-deploy-action@fa24774553152dd7873cd16ebd8d959b010c5445 # v4.9.0
with:
branch: gh-pages
folder: site
clean-exclude: pr-preview
force: false
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,11 @@ awa-python/python/awa/lib_awa.dylib.dSYM/
/awa-ui/static/assets
/awa-ui/static/index.html

# Documentation build and browser QA artifacts
/site/
/.playwright-cli/
/output/playwright/

# Local worktrees and generated artifacts
/.claude/
/artifacts/
Expand Down
51 changes: 46 additions & 5 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,13 +31,54 @@ exit code from a piped or backgrounded command.

## Schema Migrations

Migrations are forward-only and must stay rolling-upgrade compatible. Follow the
migration checklist and rolling-upgrade policy in
[`docs/development.md`](docs/development.md#authoring-schema-migrations) and
[ADR-041](docs/adr/041-rolling-upgrade-policy.md) before opening a migration PR.
Version floors, exclusive migrations, and the newer-schema fail-safe live in
Migrations are forward-only and must stay rolling-upgrade compatible. Version
floors, exclusive migrations, and the newer-schema fail-safe live in
`awa-model/src/migrations.rs`.

The implementation checklist lives here rather than in `docs/`: it is
contributor-internal, and `docs/` is published as the public documentation site.
[`docs/development.md`](docs/development.md#authoring-schema-migrations) carries
the user-facing summary and points back here.

Policy: [ADR-041 — rolling-upgrade policy](docs/adr/041-rolling-upgrade-policy.md). Use this checklist before opening a migration PR; version floors, exclusive migrations, and the newer-schema fail-safe live in `awa-model/src/migrations.rs`.

Checklist for any new `awa-model/migrations/vNNN_*.sql`:

**Every migration**

- [ ] Keep every object used by N−1 binaries compatible: no drops, type changes, or tightened constraints; make new objects and columns additive.
- [ ] Make the migration safe to re-run: `IF NOT EXISTS` on `CREATE TABLE` / `SEQUENCE` / `INDEX`, `CREATE OR REPLACE` for functions and views, `DROP TRIGGER IF EXISTS` before each `CREATE TRIGGER`, guarded `DO` blocks for anything with no `IF NOT EXISTS` form (`CREATE TYPE`), and `ON CONFLICT (version) DO NOTHING` on the `awa.schema_version` row. `migrations::tests::every_migration_guards_its_ddl` enforces the top-level cases; `test_every_migration_is_individually_re_runnable` proves it against a real database.
- [ ] Keep every step transaction-safe — the runner applies the whole pending range in one transaction, so no `CREATE INDEX CONCURRENTLY`, `VACUUM`, or statement-level `BEGIN` / `COMMIT` / `ROLLBACK` / `SAVEPOINT`. `migrations::tests::every_migration_step_is_transaction_safe` enforces this.
- [ ] In the header, link the issue and state how N−1 binaries operate against the migrated schema.
- [ ] Safe under live load: no long `ACCESS EXCLUSIVE` holds on hot tables; note the expected wall time on realistic data volumes.
- [ ] The current binary remains operable before migration, or startup applies the migration before any changed path runs. Test binary-first as well as migrate-first ordering.
- [ ] Document requirements for external runners, which do not execute Rust preflights.

**If compatibility first ships in an earlier-release patch**

- [ ] Add the released, verified patch to `MIGRATION_RUNTIME_VERSION_FLOORS`; test old, unparseable, and stale runtimes plus `--allow-live-runtimes`.
- [ ] Keep the preflight race-free and record any observability-snapshot stall from its lock. Job and lease heartbeats must remain unaffected.
- [ ] Publish the patch prerequisite before the migration and document it in the CHANGELOG and upgrade guide.

**If it changes an on-disk representation or hot-path structure (expand → flip → contract)**

- [ ] Make the migration the **expand** phase only: seed the new representation, keep the old one authoritative, and store authority explicitly. Fresh installs may start on the new representation.
- [ ] Gate the runtime **flip** on fresh fleet capability. Install the schema-owned per-feature capability constant with the expand migration; treat missing or unparseable evidence as incapable and make any override explicit.
- [ ] Under the old-writer locks, the flip treats the old representation as source of truth, reconciles the complete new representation, verifies exact equivalence, and changes authority atomically. Shadow writes alone do not satisfy this requirement.
- [ ] The flip **fences** returning pre-flip binaries at the database boundary. Exercise the actual N−1 write path; a sentinel is insufficient if old code can advance through it.
- [ ] The **contract** migration (dropping the old representation) is deferred to a later minor, tracked as its own issue, and independently checked against that release's N−1 contract.
- [ ] Model mixed-version interleavings in TLA+ when a state machine or lock order changes.
- [ ] Rehearse migrate-first, binary-first, and overlapping rollouts with a released N−1 artifact. Include concurrent old/new workers, failures and retries, scheduled work, in-flight work, hard-kill and deadline rescue, flip/fence behavior, and exact job accounting; record the evidence. CI automation is [#427](https://github.com/hardbyte/awa/issues/427).

**If no rolling-compatible design is practical**

- [ ] Explain in an ADR why expand/flip/contract and a version floor are insufficient, then add the migration to `EXCLUSIVE_WINDOW_MIGRATIONS` with refusal, override, and stale-heartbeat tests plus explicit operator documentation.

**Docs**

- [ ] Update the CHANGELOG, the release upgrade guide when operator action is required, and `docs/stability.md` when the skew contract changes. Link compatibility claims to rehearsals of the claimed version topology; describe narrower evidence only by the behavior it covers.


## Agent Skills

Canonical, portable [Agent Skills](https://agentskills.io/) live under
Expand Down
26 changes: 14 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,10 +92,10 @@ See [docs/positioning.md](docs/positioning.md) for the category map and messagin
## Getting Started

```bash
# 1. Install
pip install 'awa-pg[ui]' # Python SDK + dashboard binary
# pip install awa-pg # SDK only (no dashboard, smaller wheel)
# or: cargo add awa # Rust
# 1. Add Awa to a Python project
uv add 'awa-pg[ui]==0.6.6' # Python SDK + dashboard binary
# uv add awa-pg==0.6.6 # SDK only (no dashboard, smaller wheel)
# or: cargo add awa@0.6.6 # Rust

# 2. Start Postgres and run migrations
awa --database-url $DATABASE_URL migrate
Expand Down Expand Up @@ -310,13 +310,13 @@ Cancellation is cooperative for running handlers:
### Python

```bash
pip install awa-pg # SDK: insert, worker, admin, progress
pip install 'awa-pg[ui]' # SDK + bundled `awa` binary for the dashboard
uv add awa-pg==0.6.6 # SDK: insert, worker, admin, progress
uv add 'awa-pg[ui]==0.6.6' # SDK + bundled `awa` binary for the dashboard
# or, just the CLI:
pip install awa-cli # CLI on its own: migrations, queue admin, web UI
uv tool install awa-cli==0.6.6 # CLI on its own: migrations, queue admin, web UI
```

`pip install awa-pg` stays small for workers and producers. The `[ui]` extra pulls in [`awa-cli`](https://pypi.org/project/awa-cli/), which ships the `awa` binary plus the embedded React dashboard; afterwards `python -m awa serve` (or `awa serve` directly) launches it.
`uv add awa-pg==0.6.6` stays small for workers and producers. The `[ui]` extra pulls in [`awa-cli`](https://pypi.org/project/awa-cli/), which ships the `awa` binary plus the embedded React dashboard; afterwards `uv run python -m awa serve` (or `awa serve` directly) launches it.

### Rust

Expand All @@ -327,10 +327,10 @@ awa = "0.6"

### CLI

Available via pip (no Rust toolchain needed) or cargo:
Available as a uv tool (no Rust toolchain needed) or through cargo:

```bash
pip install awa-cli
uv tool install awa-cli==0.6.6
# or: cargo install awa-cli

awa --database-url $DATABASE_URL migrate
Expand Down Expand Up @@ -376,15 +376,17 @@ All coordination through Postgres. The Rust runtime owns dispatch, leases, heart
| `awa-worker` | Runtime: dispatch, heartbeat, maintenance |
| `awa-ui` | Web UI (axum API + embedded React frontend) |
| `awa-cli` | CLI binary (migrations, admin, serve) |
| `awa-python` | PyO3 extension module (`pip install awa-pg`) |
| `awa-python` | PyO3 extension module (`uv add awa-pg==0.6.6`) |
| `awa-testing` | Test helpers (`TestClient`) |

## Documentation

**[Browse the documentation site →](https://hardbyte.github.io/awa/)**

| Doc | Description |
| --- | --- |
| [Rust getting started](docs/getting-started-rust.md) | From `cargo add` to a job reaching `completed` |
| [Python getting started](docs/getting-started-python.md) | From `pip install` to a job reaching `completed` |
| [Python getting started](docs/getting-started-python.md) | From `uv init` to a job reaching `completed` |
| [Deployment guide](docs/deployment.md) | Docker, Kubernetes, pool sizing, graceful shutdown |
| [Migration guide](docs/migrations.md) | Fresh installs, upgrades, extracted SQL, rollback strategy |
| [0.5 → 0.6 upgrade](docs/upgrade-0.5-to-0.6.md) | Step-by-step operator checklist for the staged storage transition |
Expand Down
46 changes: 39 additions & 7 deletions awa-python/examples/quickstart.py
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
"""Awa Python quickstart — a complete runnable example.

Requires: pip install awa-pg
Requires: uv add awa-pg==0.6.6
Requires: a running Postgres instance with DATABASE_URL set.

Usage:
DATABASE_URL=postgres://localhost/mydb python examples/quickstart.py
Usage from the repository's awa-python directory:
DATABASE_URL=postgres://localhost/mydb uv run python examples/quickstart.py
"""

import asyncio
Expand Down Expand Up @@ -44,11 +44,43 @@ async def handle_email(job):
)
print(f"Inserted job {job.id} (kind={job.kind}, state={job.state})")

await asyncio.sleep(1)
await client.shutdown()
# Verify it reaches a terminal state without relying on a fixed delay.
loop = asyncio.get_running_loop()
deadline = loop.time() + 10
last_state = job.state
try:
while True:
remaining = deadline - loop.time()
if remaining <= 0:
raise TimeoutError(
f"timed out waiting for job {job.id} "
f"(last state: {last_state})"
)

# get_job is a single read-only query, so cancelling this await
# cannot leave an application transaction partially committed.
try:
result = await asyncio.wait_for(
client.get_job(job.id), timeout=remaining
)
except asyncio.TimeoutError as error:
raise TimeoutError(
f"timed out waiting for job {job.id} "
f"(last state: {last_state})"
) from error

last_state = result.state
if result.state == awa.JobState.Completed:
break
if result.state in (awa.JobState.Failed, awa.JobState.Cancelled):
raise RuntimeError(
f"job {result.id} ended in terminal state {result.state}"
)
await asyncio.sleep(min(0.1, max(0, deadline - loop.time())))
finally:
await client.shutdown()
await client.close()

# Verify it completed
result = await client.get_job(job.id)
print(f"Job {result.id} state: {result.state}")


Expand Down
Loading
Loading