Skip to content

Update signature status for hatchet workflow lifecycle #118

Description

@yedidyakfir

Summary

Currently the SignatureStatus is updated at the individual task level via SignatureLifecycle.start_task (libs/mageflow/mageflow/lifecycle/signature.py:25-29), which sets the signature to ACTIVE when a task begins executing. However, when a MageflowWorkflow is dispatched with with_signature support (see related #116), the workflow itself has no hook to transition the attached signature's status as the workflow progresses.

This issue tracks adding status updates for hatchet workflow-level signatures — in particular, marking the signature as ACTIVE when the workflow starts running, and ensuring terminal transitions (DONE / FAILED) are handled at workflow completion when the signature is attached at the workflow level rather than per-task.

Features / Details

  • Transition signature to SignatureStatus.ACTIVE when a MageflowWorkflow run begins (not only when an inner task begins)
  • Transition signature to DONE / FAILED / INTERRUPTED based on the workflow's terminal state when the signature is workflow-level
  • Respect existing should_run / is_done / is_canceled guards on TaskStatus so retried or already-completed workflows do not double-transition
  • Integrate with the existing MageWorkflow success/failure hooks (libs/mageflow/mageflow/clients/hatchet/workflow.py:29-95) so workflow-level signature updates reuse the same lifecycle machinery as task-level updates
  • Must be compatible with workflow-level with_signature (Support with_signature for hatchet workflow requests #116) — the signature carried on the workflow input is the one whose status is updated

Use Cases

  • Observability: a caller that dispatched a workflow with a signature can observe the signature transitioning PENDING → ACTIVE → DONE/FAILED without needing to inspect individual inner tasks
  • Cancellation / suspension: workflow runners can honor CANCELED / SUSPENDED states set externally before the workflow begins real work
  • Idempotency: re-runs of an already DONE workflow signature skip side effects (consistent with existing is_done checks used in task_success / task_failed)

Implementation Considerations

  • MageflowWorkflow._serialize_input (libs/mageflow/mageflow/clients/hatchet/workflow.py:109-122) is the natural place to ensure the signature data is carried into the workflow input so a workflow-start hook can resolve and update it
  • A workflow-level analogue of SignatureLifecycle.start_task is likely needed — either a dedicated WorkflowSignatureLifecycle or an extension of SignatureLifecycle that knows the unit of work is a workflow
  • The existing MageWorkflow.inject_hooks / on_success_task / on_failure_task wrappers already create a TaskSignature.ClientAdapter lifecycle per task; a parallel path is needed for the workflow-level signature (likely a first-task "start" hook or use of hatchet on_start if available, plus the existing success/failure hooks)
  • Must not regress the current task-level lifecycle behavior or the idempotency fixes recently landed (070f2d4, f58d2b3, 893d8ac)
  • Add / extend the idempotency tests under libs/mageflow/tests/unit/idempotency/ to cover workflow-level signature transitions

Tasks

  • Design where the workflow-start hook lives (new lifecycle class vs. extension of SignatureLifecycle)
  • Implement workflow-start status transition (PENDING → ACTIVE) for MageflowWorkflow
  • Implement workflow-terminal status transitions (DONE / FAILED) for workflow-level signatures
  • Guard transitions with should_run / is_done to preserve idempotency on retries
  • Wire workflow-level lifecycle into MageWorkflow alongside the existing task-level hooks
  • Add unit tests for workflow-level signature status transitions
  • Add integration test: dispatch a MageflowWorkflow with a signature and assert PENDING → ACTIVE → DONE
  • Add idempotency test: re-running a DONE workflow signature does not re-trigger callbacks
  • Update docs / examples if workflow-level signature status is user-visible

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions