Openframe Client Updater - #1783
Open
danylo-babenko-flamingo wants to merge 10 commits into
Open
Conversation
…en logic New OS service (com.openframe.client-updater) that owns the openframe-client binary swap — a second process is required because Windows locks a running executable. Reuses the client's shared_token.enc when available and falls back to its own OAuth2 client_credentials refresh from agent_config.json; tokens are held in memory only.
Crate part of oss-tenant 3a409150f; that commit's CI matrix entries were oss-tenant's workflow and land here in a separate CI commit.
…to-rollback Update phases run Downloading through VerifyingBoot and Observing. The new binary must write a version-matching boot marker within 90s, then survive a 10-minute observation window; failure at either gate rolls back to the last-known-good reserve. Promotion to the LKG anchor happens only after observation passes.
…s from reserve A message carrying rollback: true is allowed to downgrade; when the target version matches the .lkg reserve it restores offline without a download.
…ng calls, secured-dir perms Durable state writes, corrected JetStream ACK semantics, blocking filesystem calls moved off the async runtime, and hardened secured-directory permissions.
Sidecar *_tests.rs per module: updater state machine, atomic replace, encryption round-trip, versions_match, archive extraction, and phase-string parity with the client's probe contract.
…recovery Applies the 11 external review findings: redelivered duplicates can no longer cancel a live observation window; the backup path is precomputed so a failed replace cannot brick the install; macOS service stop tolerates not-loaded and polls for real termination; crash recovery runs before token wait and NATS connect; promotion requires the service to actually be running; dedicated 300s client download timeout; ACK wait raised to 600s; +crt-static for Windows builds.
oss-lib CI enforces cargo fmt --check; the crate arrived verbatim from oss-tenant, which is unformatted.
… client swap The updater now owns CLIENT_UPDATE end to end: removes the in-client update service, state machine, platform update scripts, and updater launcher. Tool listeners route through park_or_dispatch, which probes the updater's state file and holds messages with AckKind::Progress keep-alives every 60s while a swap is in flight, preserving the JetStream redelivery budget until the new client processes them after restart. Also adds the updater version flag and updater log source. Adapts oss-tenant commits 3813b1443, 6a3851a90, 9afd7f8df, 3346dba1e, and 012127cdb to the agent-lib layout.
Adds the updater's Makefile lint/build/test steps to the test-rust matrix (macOS + Windows), matching the client's steps.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Port openframe-client-updater from oss-tenant
Ports the complete client-updater v2 work from
openframe-oss-tenant(branchhotfix/create-client-updater-service-v2, PR #2180) into oss-lib as the new home of client development. The updater crate arrives verbatim from the oss-tenant branch tip (rustfmt-applied to satisfy this repo'scargo fmt --checkCI), and the matching client-side changes are replayed ontoopenframe-agent-lib.What this adds
New crate:
clients/openframe-client-updaterA standalone OS service (
com.openframe.client-updater) that owns the openframe-client binary swap. A second process is required because Windows locks a running executable — the client cannot replace itself. The updater owns theCLIENT_UPDATEJetStream consumer end to end; the client no longer participates in its own update.Hybrid token handling. The updater first reuses the client's token (
shared_token.enc, AES-GCM); if the client is down or the token is unusable, it falls back to its own OAuth2client_credentialsrefresh using the sharedagent_config.jsoncredentials. Tokens live in memory only.Last-known-good ratchet with health-gated promotion. Each update walks
Downloading → Verifying → StoppingService → ReplacingBinary → StartingService → VerifyingBoot → Observing:.lkgreserve copy of the binary kept next to the executable.A failed update can therefore only fall back to the last verified version, never an arbitrarily old backup.
Rollback support. An incoming message with
rollback: truebypasses the downgrade guard and, when the target version matches the reserve, restores it offline — no download needed.Crash recovery. On startup the updater inspects persisted state (
updater_state.json) and finishes whatever a mid-update crash interrupted — completing the swap, verifying boot, or rolling back — before connecting to NATS; the outcome is reported once connected.Client-side (
openframe-agent-lib)In-client update machinery removed (−3,400 lines): the old update service, state machine, PowerShell/shell update scripts, and updater launcher are gone. The client keeps exactly one update-related duty: installing and updating the updater itself as a tool.
Update ordering. During a client swap, all tool operations (install / uninstall / tool update / restart) are parked, not dropped:
park_or_dispatchprobes the updater's state file and, while a swap is in flight, holds each message withAckKind::Progresskeep-alives every 60s. Progress acks preserve the full JetStream redelivery budget, so a parked message survives an arbitrarily long update and is processed by the new client after restart. The reverse guard also holds: the updater defers a client swap while a tool operation is running (30-minute staleness cutoff protects against a stuck state file).Migration safety. The updater uses a fresh durable/inbox identity (
…_client-updater_client-update_consumer) withDeliverPolicy::New, so old in-place clients and the new updater never race over the same consumer during rollout.Review hardening (11 findings from external review, all applied)
Redelivered duplicates can no longer cancel a live observation window; a failed binary replace can no longer brick the install (backup path is precomputed and restored); macOS service stop tolerates already-unloaded services and polls for real termination; crash recovery runs before any network dependency; promotion requires the service to actually be running, not just a boot marker; client downloads get a dedicated 300s timeout; ACK wait raised to 600s to match real time-to-ACK; Windows builds are statically linked (
+crt-static).Tests and CI
61 unit tests in the updater crate (sidecar
*_tests.rsper module — state machine, atomic replace, encryption round-trip, version matching, archive extraction, phase-string parity with the client's probe contract) alongside the client's 108. CI (test.yml) lints, builds, and tests the updater on macOS and Windows next to the client.Commit guide
The updater-crate commits replay the oss-tenant branch history snapshot by snapshot (each state compiled and passed review there), so the crate's evolution is reviewable commit by commit:
feat(client-updater): port standalone updater service with hybrid token logic— the service skeleton, NATS listener, token handlingchore(client-updater): clippy and fmt cleanupsfeat(client-updater): LKG ratchet, boot-marker verify, observation auto-rollback— the update state machinefeat(client-updater): rollback flag bypasses downgrade guard, restores from reservefix(client-updater): review fixes — durability, ACK semantics, blocking calls, secured-dir permstest(client-updater): unit tests for models, platform, and servicesfix(client-updater): harden update lifecycle, binary swap, and crash recovery— the 11 external review findingsstyle(client-updater): rustfmt— formatting only, to satisfy this repo'sfmt --checkCIfeat(client): remove in-client update machinery, park tool ops during client swap— the client side, adapted to the agent-lib layout (the oss-tenant client commits were written against the old in-tenant client structure, so they land here as one adapted commit)ci: lint, build, and test openframe-client-updaterOriginal per-commit diffs: oss-tenant PR #2180.
Validation
clippy -D warningsclean,fmt --checkclean--features binFollow-ups (not in this PR)
openframe-client-updaterbin + release matrix entry (mirrors the client's thin-bin pattern)