Shut down workers on idle instead of a static maximum lifetime

[conradbzura]

Description

Replace the static CFDB_WORKER_MAX_LIFETIME_SECONDS wall-clock ceiling with idle-based self-termination: a worker should exit when it has been idle beyond a configurable threshold, using the idle_time RPC endpoint wool introduced in v0.14.0-rc0 (wool-labs/wool#269).

Today worker_main self-terminates after a fixed lifetime (default 5 h) because wool exposed no per-job activity signal — the worker could not tell idle from busy, so a hard ceiling was the only way to keep over-provisioned workers from accumulating (the docstring at worker_main.py:78 records this). The idle_time endpoint removes that constraint: the worker can now observe how long it has been without in-flight work and exit only when genuinely idle.

This requires bumping the wool pin from ~=0.13.0 to the 0.14 line.

Motivation

• Mid-task preemption. A static lifetime fires regardless of whether the worker is mid-task, and the max-lifetime exit path deliberately skips the drain-grace window the SIGTERM path provides — so a worker that picks up a job late in its lifetime can be killed while work is in flight. Idle-based shutdown makes this impossible by construction: a busy worker's idle clock is zero.
• Faster reclamation of idle capacity. The priority load balancer intentionally drains over-provisioned workers to idle, but they then bill Fargate time for up to 5 h before self-reaping. An idle threshold on the order of minutes reclaims that capacity almost immediately.
• Shrinks the wool-version-bump wedge. Workers rejected by a new API after a wool version bump sit idle but still count toward ECS_MAX_WORKERS until max-lifetime reaps them (documented in the README's moving-tag caveats). Idle-based shutdown reaps them within the idle threshold instead of hours later.

Expected Outcome

• Workers self-terminate after a configurable idle threshold (e.g. CFDB_WORKER_IDLE_TIMEOUT_SECONDS) measured via wool's idle_time endpoint, rather than after a fixed wall-clock lifetime.
• A worker with in-flight work is never self-terminated; idle-triggered shutdown goes through the existing drain path (unhealthy /health, graceful stop) so discovery and ECS observe the exit cleanly.
• The wool dependency is bumped to a version carrying the idle_time endpoint (≥ 0.14.0), with the API↔worker rollout ordering considered per the README's version-skew caveats.
• CFDB_WORKER_MAX_LIFETIME_SECONDS is either removed or retained as a loose safety backstop (e.g. against a stuck job holding a slot indefinitely) — to be decided during implementation; provisioner.py:471 currently names it the fleet's only reaping safety net.