Skip to content

Openframe Client Updater - #1783

Open
danylo-babenko-flamingo wants to merge 10 commits into
mainfrom
hotfix/create-client-updater-service-v2
Open

Openframe Client Updater#1783
danylo-babenko-flamingo wants to merge 10 commits into
mainfrom
hotfix/create-client-updater-service-v2

Conversation

@danylo-babenko-flamingo

@danylo-babenko-flamingo danylo-babenko-flamingo commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Port openframe-client-updater from oss-tenant

Ports the complete client-updater v2 work from openframe-oss-tenant (branch hotfix/create-client-updater-service-v2, PR #2180) into oss-lib as the new home of client development. The updater crate arrives verbatim from the oss-tenant branch tip (rustfmt-applied to satisfy this repo's cargo fmt --check CI), and the matching client-side changes are replayed onto openframe-agent-lib.

What this adds

New crate: clients/openframe-client-updater

A standalone OS service (com.openframe.client-updater) that owns the openframe-client binary swap. A second process is required because Windows locks a running executable — the client cannot replace itself. The updater owns the CLIENT_UPDATE JetStream consumer end to end; the client no longer participates in its own update.

Hybrid token handling. The updater first reuses the client's token (shared_token.enc, AES-GCM); if the client is down or the token is unusable, it falls back to its own OAuth2 client_credentials refresh using the shared agent_config.json credentials. Tokens live in memory only.

Last-known-good ratchet with health-gated promotion. Each update walks Downloading → Verifying → StoppingService → ReplacingBinary → StartingService → VerifyingBoot → Observing:

  • the new binary must write a boot marker within 90s whose version matches the target (semver-normalized);
  • a 10-minute observation window follows — a crash or failed health check inside it triggers automatic rollback;
  • only after surviving observation is the version promoted to the last-known-good anchor, with a .lkg reserve copy of the binary kept next to the executable.

A failed update can therefore only fall back to the last verified version, never an arbitrarily old backup.

Rollback support. An incoming message with rollback: true bypasses the downgrade guard and, when the target version matches the reserve, restores it offline — no download needed.

Crash recovery. On startup the updater inspects persisted state (updater_state.json) and finishes whatever a mid-update crash interrupted — completing the swap, verifying boot, or rolling back — before connecting to NATS; the outcome is reported once connected.

Client-side (openframe-agent-lib)

In-client update machinery removed (−3,400 lines): the old update service, state machine, PowerShell/shell update scripts, and updater launcher are gone. The client keeps exactly one update-related duty: installing and updating the updater itself as a tool.

Update ordering. During a client swap, all tool operations (install / uninstall / tool update / restart) are parked, not dropped: park_or_dispatch probes the updater's state file and, while a swap is in flight, holds each message with AckKind::Progress keep-alives every 60s. Progress acks preserve the full JetStream redelivery budget, so a parked message survives an arbitrarily long update and is processed by the new client after restart. The reverse guard also holds: the updater defers a client swap while a tool operation is running (30-minute staleness cutoff protects against a stuck state file).

Migration safety. The updater uses a fresh durable/inbox identity (…_client-updater_client-update_consumer) with DeliverPolicy::New, so old in-place clients and the new updater never race over the same consumer during rollout.

Review hardening (11 findings from external review, all applied)

Redelivered duplicates can no longer cancel a live observation window; a failed binary replace can no longer brick the install (backup path is precomputed and restored); macOS service stop tolerates already-unloaded services and polls for real termination; crash recovery runs before any network dependency; promotion requires the service to actually be running, not just a boot marker; client downloads get a dedicated 300s timeout; ACK wait raised to 600s to match real time-to-ACK; Windows builds are statically linked (+crt-static).

Tests and CI

61 unit tests in the updater crate (sidecar *_tests.rs per module — state machine, atomic replace, encryption round-trip, version matching, archive extraction, phase-string parity with the client's probe contract) alongside the client's 108. CI (test.yml) lints, builds, and tests the updater on macOS and Windows next to the client.

Commit guide

The updater-crate commits replay the oss-tenant branch history snapshot by snapshot (each state compiled and passed review there), so the crate's evolution is reviewable commit by commit:

  1. feat(client-updater): port standalone updater service with hybrid token logic — the service skeleton, NATS listener, token handling
  2. chore(client-updater): clippy and fmt cleanups
  3. feat(client-updater): LKG ratchet, boot-marker verify, observation auto-rollback — the update state machine
  4. feat(client-updater): rollback flag bypasses downgrade guard, restores from reserve
  5. fix(client-updater): review fixes — durability, ACK semantics, blocking calls, secured-dir perms
  6. test(client-updater): unit tests for models, platform, and services
  7. fix(client-updater): harden update lifecycle, binary swap, and crash recovery — the 11 external review findings
  8. style(client-updater): rustfmt — formatting only, to satisfy this repo's fmt --check CI
  9. feat(client): remove in-client update machinery, park tool ops during client swap — the client side, adapted to the agent-lib layout (the oss-tenant client commits were written against the old in-tenant client structure, so they land here as one adapted commit)
  10. ci: lint, build, and test openframe-client-updater

Original per-commit diffs: oss-tenant PR #2180.

Validation

  • updater crate: 61 tests green, clippy -D warnings clean, fmt --check clean
  • agent-lib: 108 tests green, clippy + fmt clean for both the lib and --features bin
  • crate content verified byte-identical to the oss-tenant branch tip before formatting

Follow-ups (not in this PR)

  • saas-tenant thin openframe-client-updater bin + release matrix entry (mirrors the client's thin-bin pattern)
  • registry JSON download links (oss-tenant PR #2254) must move from oss-tenant releases to saas-tenant releases once the updater ships from there

…en logic

New OS service (com.openframe.client-updater) that owns the openframe-client binary swap — a second process is required because Windows locks a running executable. Reuses the client's shared_token.enc when available and falls back to its own OAuth2 client_credentials refresh from agent_config.json; tokens are held in memory only.
Crate part of oss-tenant 3a409150f; that commit's CI matrix entries were oss-tenant's workflow and land here in a separate CI commit.
…to-rollback

Update phases run Downloading through VerifyingBoot and Observing. The new binary must write a version-matching boot marker within 90s, then survive a 10-minute observation window; failure at either gate rolls back to the last-known-good reserve. Promotion to the LKG anchor happens only after observation passes.
…s from reserve

A message carrying rollback: true is allowed to downgrade; when the target version matches the .lkg reserve it restores offline without a download.
…ng calls, secured-dir perms

Durable state writes, corrected JetStream ACK semantics, blocking filesystem calls moved off the async runtime, and hardened secured-directory permissions.
Sidecar *_tests.rs per module: updater state machine, atomic replace, encryption round-trip, versions_match, archive extraction, and phase-string parity with the client's probe contract.
…recovery

Applies the 11 external review findings: redelivered duplicates can no longer cancel a live observation window; the backup path is precomputed so a failed replace cannot brick the install; macOS service stop tolerates not-loaded and polls for real termination; crash recovery runs before token wait and NATS connect; promotion requires the service to actually be running; dedicated 300s client download timeout; ACK wait raised to 600s; +crt-static for Windows builds.
oss-lib CI enforces cargo fmt --check; the crate arrived verbatim from oss-tenant, which is unformatted.
… client swap

The updater now owns CLIENT_UPDATE end to end: removes the in-client update service, state machine, platform update scripts, and updater launcher. Tool listeners route through park_or_dispatch, which probes the updater's state file and holds messages with AckKind::Progress keep-alives every 60s while a swap is in flight, preserving the JetStream redelivery budget until the new client processes them after restart. Also adds the updater version flag and updater log source. Adapts oss-tenant commits 3813b1443, 6a3851a90, 9afd7f8df, 3346dba1e, and 012127cdb to the agent-lib layout.
Adds the updater's Makefile lint/build/test steps to the test-rust matrix (macOS + Windows), matching the client's steps.
@danylo-babenko-flamingo danylo-babenko-flamingo changed the title feat(clients): port openframe-client-updater from oss-tenant Openframe Client Updater Aug 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant