feat(runs): report a live phase for every automation run - #353
Open
aartem1 wants to merge 8 commits into
Open
Conversation
5 tasks
Adds phase_code/phase_label/phase_updated_at to AutomationRun and a
POST /v1/runs/{run_id}/phase endpoint so code running inside the sandbox
can report what it is doing. The endpoint mirrors complete_run: same
credentials, same automation ownership check.
A phase is one value -- the (code, label) pair -- and every successful
write replaces it whole, so a field omitted from the request is stored as
NULL rather than keeping its previous value. Merging would let a stale
code sit beside a fresh label as a pair nobody ever wrote.
Validation rejects with 422 before any write, leaving run status and the
stored phase untouched: label at most 200 characters, at least one of
code/label non-blank, and no Unicode Cc/Zl/Zp characters in either field.
Cf is deliberately not rejected -- it would ban the zero-width joiners
inside emoji sequences and the bidi marks non-Latin labels carry.
Segment S01 of tasks/IMPL-automation-run-phases.md.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Between accepting a run and starting its entrypoint the service was a blind spot: the run sat in RUNNING while a sandbox was provisioned and a bundle uploaded, and nothing told a hung run from a working one. The dispatcher and the shared execution path now record four milestones -- queued, sandbox_provisioning, bundle_upload, entrypoint_start -- which the frontend translates. They are written through _record_run_phase, which writes the columns directly rather than calling the service's own REST endpoint, and which never raises: a phase is telemetry, so failing to record one must not fail the run. Writing stops at entrypoint_start so the service cannot stamp over a phase the automation itself reports afterwards. Both call sites sit in code shared by the cloud and local backends, so the milestones exist in sandboxless mode too. Also injects AUTOMATION_PHASE_URL beside AUTOMATION_CALLBACK_URL. The callback variable turned out to be the full /complete URL rather than a base, so automation code had no address to build the phase endpoint from, and chopping the suffix off would be brittle. Segment S02 of tasks/IMPL-automation-run-phases.md. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Once the entrypoint starts, the service stops writing phases and the run
goes dark for the rest of its life -- which is where the interesting work
happens. Both presets now report "preparing" and "running_agent" through
POST /v1/runs/{run_id}/phase, and docs/run-phase-reporting.md gives the
author of a custom automation the same recipe.
Reporting is best-effort: a missing AUTOMATION_PHASE_URL, a network
error, a timeout or a non-2xx response are all swallowed, and the request
carries a timeout so a hung endpoint cannot stall the automation.
The credential is AUTOMATION_CALLBACK_API_KEY in local mode and
OPENHANDS_API_KEY in Cloud -- the two the backends actually inject.
SESSION_API_KEY is deliberately not used despite being present in the
sandbox: it authenticates to the agent server, a different service, and
the local automation service accepts only its own key, so sending it
would 401 and drop every phase silently.
Preset templates are never executed by the suite, so the tests assert on
the module AST instead of source text: every call site's literal
arguments are fed to the real RunPhaseRequest, each call is checked to be
unnested, and the urllib import is checked to exist -- a deleted import
would otherwise raise inside the helper's own try and be swallowed.
Segment S03 of tasks/IMPL-automation-run-phases.md.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
aartem1
force-pushed
the
feat/automation-run-phases
branch
from
August 20, 2026 09:49
60a234a to
876a737
Compare
A phase write was accepted whatever the run's status, so a sandbox that outlived its run — or anything else holding the same credential — could move the phase of a run that had already finished. For a failed run that phase is the record of where it stopped, and the UI shows it beside the failure, so overwriting it destroys the only account of how far the run got. Only PENDING and RUNNING runs now accept a phase; anything else answers 409, the same way `cancel_run` refuses a terminal run. The status is re-checked in the UPDATE itself, because the completion callback can land between the check and the write. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The contract lets a phase carry only a code, and the UI shows that code raw when it does — worth knowing before shipping `checking_out` to a user's screen. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The label is stored as sent — only a phase blank on both fields is rejected — so the UI trims it and falls back to the code rather than rendering blank text. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
|
👋 This PR needs a couple of things fixed before OpenHands can review it:
Push an update once this is addressed and this check re-runs automatically. This is an automated check - no AI was used to generate this comment. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
HUMAN:
I checked this by hand on a local stack. First the cases in the screenshots on
the frontend PR (OpenHands/OpenHands#16740) — the phase on the automations
list, on the activity-log rows, and the full label on hover — all driven by
this service. On top of that I ran a real but mocked scenario: a mock
automation whose entrypoint does no real work but reports its phases through
the new endpoint. I watched the phases arrive in order as the run progressed,
and saw a failed run keep the phase it stopped at.
AGENT:
Verified beyond unit tests: pre-commit and the full pytest suite, plus the
end-to-end local dispatch described under "Testing" below.
Why
An automation run reports one thing for its entire lifetime:
RUNNING. A runthat takes minutes looks identical at second 5 and second 300, and a run that
fails tells you that it failed, never where it stopped — provisioning, the
bundle upload, the entrypoint, or somewhere inside the automation's own code.
This adds a current phase to a run: a short machine code, a human label, and
the moment it was written. The service records its own preparation milestones;
code running inside the sandbox reports the rest through a new endpoint.
Relates to OpenHands/OpenHands#16572. Frontend side: OpenHands/OpenHands#16740
What changed
automation_runs(phase_code,phase_label,phase_updated_at) plus migration016. Nullable with nobackfill: an existing run simply has no phase, which is the truth.
POST /v1/runs/{run_id}/phase, authenticated like theexisting completion callback, 404 for an unknown run and 403 for someone
else's automation. Every write replaces the whole
(code, label)pair —last write wins, only the current phase is kept, so there is no history
table to retain or prune. At least one of
code/labelis required.queued,sandbox_provisioning,bundle_uploadandentrypoint_startas itprepares a run. A failed phase write never fails the run: a phase is
telemetry, not control flow.
promptandplugintemplates reportpreparingand
running_agentfrom inside the sandbox, using the same endpoint anycustom automation can use.
docs/run-phase-reporting.mdis the recipe for customentrypoints, including which credential to use.
The credential subtlety worth a reviewer's attention
The sandbox holds two keys.
SESSION_API_KEYauthenticates to the agentserver; the automation service rejects it. Phase reports must use
AUTOMATION_CALLBACK_API_KEY(local mode) orOPENHANDS_API_KEY(cloud).Getting this wrong is silent — the POST 401s, the caller swallows it because a
phase must never break a run, and phases simply never appear while every test
stays green. The preset templates and the docs both spell this out.
Testing
uv run pre-commit run --all-files— clean (yamlfmt, ruff format, rufflint, pycodestyle, pyright).
uv run pytest tests/— 1337 passed. 36 of those are new: the endpoint'scontract (auth, ownership, validation, replacement semantics), the
dispatcher milestones against a real DB seam, and AST assertions that both
preset templates emit the phases with the right credential.
whose entrypoint is a shell script that does no real work but reports real
phases through the endpoint. Dispatching it walks
queued → sandbox_provisioning → bundle_upload → entrypoint_start → preparing → …and finishesCOMPLETEDin ~56s; a second variant thatreports
FAILEDthrough the completion callback keeps its last phasevisible on the failed run, which is the point of the feature.