Summary
Add idempotency support for signature creation within workflow durable tasks. Currently, the signature retry cache only works for standalone durable tasks (@hatchet.durable_task()). Workflows containing durable tasks need the same idempotency guarantees — ensuring that on retry, signatures are not duplicated regardless of whether the workflow itself is signed or unsigned.
Current Behavior
- Standalone durable tasks: Retry cache is automatically set up via
handle_task_callback(is_idempotent=True), preventing duplicate signature creation on retries
- Workflow durable tasks: No retry cache setup — if a durable task inside a workflow retries,
asign(), achain(), aswarm() will create duplicate signatures
- Signed workflows: Treated as
TaskSignature via asign(), but inner durable tasks don't inherit idempotency
Desired Behavior
- Durable tasks inside workflows should automatically get retry cache support, just like standalone durable tasks
- Signed workflows should propagate idempotency context to their inner tasks
- Unsigned workflows with durable tasks should also support retry cache for signature creation within those tasks
- Signature creation order must remain deterministic across retries (existing constraint from
docs/documentation/idempotency.md)
Use Cases
- A workflow with a durable task that calls
asign() retries after failure → should return the cached signature, not create a new one
- A signed workflow dispatched via
asign(workflow) contains durable tasks that themselves create signatures → all inner signatures should be idempotent on retry
- An unsigned workflow registered via
@mageflow.workflow() has durable tasks → those tasks' signatures should be cached
Technical Considerations
- Retry cache is keyed by
workflow_id (Hatchet's task run ID) — need to determine the correct key scope for workflow-level tasks
_inject_workflow_hooks() in HatchetMageflow already injects completion hooks — may need to also inject retry cache setup for durable tasks within workflowshandle_task_callback() controls is_idempotent flag — workflow task decoration needs to pass this through- Must handle both
task() and durable_task() within workflow context
Related
Tasks
Summary
Add idempotency support for signature creation within workflow durable tasks. Currently, the signature retry cache only works for standalone durable tasks (
@hatchet.durable_task()). Workflows containing durable tasks need the same idempotency guarantees — ensuring that on retry, signatures are not duplicated regardless of whether the workflow itself is signed or unsigned.Current Behavior
handle_task_callback(is_idempotent=True), preventing duplicate signature creation on retriesasign(),achain(),aswarm()will create duplicate signaturesTaskSignatureviaasign(), but inner durable tasks don't inherit idempotencyDesired Behavior
docs/documentation/idempotency.md)Use Cases
asign()retries after failure → should return the cached signature, not create a new oneasign(workflow)contains durable tasks that themselves create signatures → all inner signatures should be idempotent on retry@mageflow.workflow()has durable tasks → those tasks' signatures should be cachedTechnical Considerations
workflow_id(Hatchet's task run ID) — need to determine the correct key scope for workflow-level tasks_inject_workflow_hooks()inHatchetMageflowalready injects completion hooks — may need to also inject retry cache setup for durable tasks within workflowshandle_task_callback()controlsis_idempotentflag — workflow task decoration needs to pass this throughtask()anddurable_task()within workflow contextRelated
Tasks