Helix

Helix is a self-hostable durable distributed execution platform: brokered execution, external processor fabric, static DAG workflows, persistent leases, replayable events, scheduling, signals, typed SDKs, and operations tooling.

The production process hosts the HTTP control plane, lease recovery, durable outbox publishing, and the persistent schedule executor. Postgres is authoritative; Redpanda or another Kafka-compatible broker receives derived runtime events.

See PRD.md for product requirements and phased delivery.

Architecture docs

• Architecture Decision Records
• Domain glossary
• Architecture checklist
• Event policy

Workspace

Yarn workspace layout:

• apps/control-plane — self-hosted API, browser UI, authentication, and /admin operations control plane.
• services/broker — reusable job, lease, attempt, and routing policies; the production worker lifecycle is hosted by the control plane.
• services/scheduler — deterministic scheduling policy/test library; the production persistent executor is hosted by the control plane.
• packages/contracts — shared API and event contracts.
• packages/producer-sdk — TypeScript client for jobs, workflows, schedules, and event streams.
• packages/processor-sdk — TypeScript client and polling loop for external processors.
• packages/workflow-sdk — typed workflow DAG builder and validator.

Local infrastructure

Helix uses Postgres as authoritative state and Redpanda as the Kafka-compatible derived event bus for local/CI smoke checks.

cp .env.example .env
docker compose up -d postgres redpanda
yarn infra:smoke
yarn db:migrate
docker compose stop postgres redpanda

Use docker compose down -v only when you intentionally want to delete local Postgres and Redpanda volumes.

Self-host with Docker Compose

The Compose stack builds and runs the production control plane together with Postgres and Redpanda. The one-shot migrate and bootstrap services must complete successfully before app starts; failed migrations or bootstrap configuration prevent the application from starting. The application is published on port 3000 by default and exposes liveness at /health plus database readiness at /ready.

cp .env.example .env
# Replace every `replace-with-...` value and the database password in .env.
docker compose up -d --build
docker compose ps
curl --fail http://127.0.0.1:3000/health
curl --fail http://127.0.0.1:3000/ready

Open http://127.0.0.1:3000/login and sign in with HELIX_BOOTSTRAP_ADMIN_EMAIL and HELIX_BOOTSTRAP_ADMIN_PASSWORD. Bootstrap is deliberately idempotent: after the first local user exists, later starts never replace that user or its password. Create and disable additional accounts under Admin → Users/RBAC. If the administrator password is lost, reset it from the application container:

docker compose run --rm \
  -e HELIX_RESET_PASSWORD_EMAIL=admin@example.com \
  -e HELIX_RESET_PASSWORD='a-new-long-unique-passphrase' \
  app node apps/control-plane/dist/server/reset-password-cli.js

Compose binds Helix to 127.0.0.1 by default. Keep that loopback binding when a reverse proxy on the same host is the public entry point, and terminate TLS at the proxy. Set HELIX_BIND_ADDRESS=0.0.0.0 only for an intentionally exposed private network. HELIX_ALLOWED_ORIGINS accepts a comma-separated list such as https://helix.example.com,https://ops.example.com; same-origin requests do not need an entry.

To upgrade, back up the Postgres volume/database first and update the checkout. Before the first local-auth bootstrap of an existing database, set HELIX_BOOTSTRAP_TENANT_SLUG, HELIX_BOOTSTRAP_ORGANIZATION_SLUG, and HELIX_BOOTSTRAP_PROJECT_SLUG in .env to the exact slugs of one existing tenant/organization/project scope. Also keep the corresponding HELIX_BOOTSTRAP_*_NAME values descriptive. The default scope values in .env.example are only for a genuinely new, empty installation; do not use them for an existing scope with different slugs.

Then run docker compose up -d --build again. Compose runs the idempotent migration and authentication-bootstrap jobs before replacing the healthy app. A scope mismatch aborts bootstrap instead of creating a second empty scope. Inspect startup failures with docker compose logs migrate bootstrap app.

The local-auth migration preserves tenant, project, job, workflow, and event data while removing the obsolete Stytch organization field. Helix does not call an external identity provider at runtime.

Machine credentials are stored as versioned keyed HMAC-SHA256 digests using HELIX_TOKEN_SECRET_PEPPER. Legacy 64-hex PBKDF2 digests remain verifiable through a small bounded compatibility path when the installation keeps its previously configured pepper, but should be replaced by revoking and recreating project API keys and agent registration credentials after the upgrade. An older installation that never configured a private pepper used a publicly known development fallback; treat those credentials as compromised and rotate them instead of carrying that fallback forward. Changing the pepper intentionally invalidates existing project API keys, agent registration credentials, and agent tokens.

The production image runs as an unprivileged user, serves only Vite's built /assets/* files, and handles SIGTERM by stopping HTTP, scheduler, workflow/lease maintenance, Kafka publishing, and the database pool. Source files under /src are not exposed.

Production build without Docker

Node.js 22, an externally managed Postgres instance, and a Kafka-compatible broker are required for the complete runtime. Set HELIX_KAFKA_BROKERS for an external cluster, or use HELIX_REDPANDA_HOST and HELIX_REDPANDA_KAFKA_PORT.

cp .env.example .env
# Configure Postgres, Kafka, login, session, and token secrets in .env.
yarn install --immutable
yarn build
NODE_ENV=production yarn db:migrate:production
NODE_ENV=production yarn auth:bootstrap:production
NODE_ENV=production yarn start

yarn build creates the browser bundle in apps/control-plane/dist/client and the Node server plus migration and authentication CLIs in apps/control-plane/dist/server. The bootstrap command is idempotent: it creates the first local administrator once and leaves existing credentials unchanged on later starts. yarn start only runs built artifacts; it does not invoke Vite or execute TypeScript source files.

Root runtime commands (infra:smoke, migration, authentication bootstrap/reset, and start) load .env with Node's --env-file-if-exists support. Values already exported in the process environment take precedence, so CI and secret-manager injection continue to override the file.

Connect producers and processors

After signing in, create a scoped project key under API Keys for a producer. Create an agent registration credential under Processors for each processor host. Both secrets are displayed exactly once; store them in the consuming service's secret manager. A processor exchanges its registration credential at /api/v1/agents/token and then uses the returned short-lived agent token. The TypeScript producer and processor SDK READMEs contain complete examples.

Workflow recovery

Postgres-backed workflow runs, steps, dependencies, and timers are authoritative. The built-in maintenance worker resumes active runs after a restart, reconciles terminal job states, wakes due timers, and advances dependency-ready steps. Step jobs use idempotency keys of the form workflow-step:<runId>:<stepId>, so recovery does not duplicate completed or already activated work. Terminal failed or dead-lettered jobs fail their owning step and run; completed jobs advance the graph even when the original callback was interrupted by a crash.

Commands

yarn install --immutable
yarn build
yarn start
yarn docs:check
yarn tooling:check
yarn infra:smoke
yarn db:migrate
yarn db:migrate:production
yarn auth:bootstrap
yarn auth:bootstrap:production
yarn auth:reset-password
yarn auth:reset-password:production
yarn check
yarn test
yarn lint
yarn validate