Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/actions/encrypt-env/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,8 @@ asset: mainframe.sops.env
keys:
- mainframe
env:
APPS_DOMAIN: APPS_DOMAIN # output name: GitHub Secret/Variable name
APPS_TIMEZONE: APPS_TIMEZONE
DOMAIN: DOMAIN # output name: GitHub Secret/Variable name
TIMEZONE: TIMEZONE
```

For each name in `keys`, it loads `<keys-directory>/<name>.pub`. Secrets take precedence over Variables when both contain the same source key. Every source key must resolve or the action fails. The manifest has no `apps` field — app selection lives on the deploy target, not the vault; see the main README's "Vaults And Targets" section.
76 changes: 47 additions & 29 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,48 +109,66 @@ Compose files mount the rendered file by its plain relative path (`./traefik.yml

When adding config for a new app: only reach for a template if the image has no env-var-driven config path at all. If it does (most well-behaved images do), prefer plain `environment:` entries over a template - fewer moving parts, and the value never touches disk as a separate file.

### Per-app and per-server overrides
### App-specific vs. shared variables

Compose files explicitly declare which variables are overridable using bash fallback syntax:
Inside a `docker-compose.yml`/`*.tpl` file, variable names are always bare - never prefixed with the app's own name. Each app already has its own compose file and its own generated `.env`, so an app-name prefix inside it would disambiguate nothing; `${PUBLIC_KEY}` in `apps/beszel-agent/docker-compose.yml` and `${DOMAIN}` shared across a dozen apps look exactly the same from inside the file, because the file is already scoped to one app.

```yaml
# App-specific override, falls back to server-wide value
SHOWS_PATH: ${JELLYFIN_SHOWS_PATH:-${APPS_SHOWS_PATH}}
What makes a variable "shared" vs. "app-specific" is a fact about the *vault*, not the compose file: whether multiple apps' vaults map the same variable name to the same GitHub Secret/Variable, or only one app's vault ever references it at all.

# App-specific only — must be set per app
SHOWS_PATH: ${JELLYFIN_SHOWS_PATH}
App-name prefixing happens on the *other* side of a vault manifest's `env:` mapping - the GitHub Secret/Variable name - and only as a judgment call when it helps a human scanning a flat list of a target's Secrets/Variables tell which app a value belongs to (e.g. `TRAEFIK_HTTP_PORT` keeps its prefix because a bare `HTTP_PORT` wouldn't self-document; `ADMIN_MAIL` doesn't need one because it doesn't need explaining). See "Vaults And Targets" in README for the manifest format.

# Server-wide — same value for all apps on this server
SHOWS_PATH: ${APPS_SHOWS_PATH}
```
There's no per-app override mechanism (a bash-fallback `${APPNAME_VAR:-${VAR}}` pattern existed here before and was removed - nothing in the catalog ever used it). If an app genuinely needs a value another app also uses but with a different value, give it its own distinctly-named variable instead - not a namespaced variant of the same name.

Name a variable by what it *is*, never by its format (`SESSION_KEY`, not `KEY_HEX_32`) - a format-shaped name invites treating same-format secrets as interchangeable across apps when they aren't. This matters concretely for keys: a **session-signing key** (Better Auth, Django's `SECRET_KEY` - only ever used to sign/verify sessions and CSRF tokens) is safe to genuinely share across apps, since compromising it just forces every app's active sessions to re-authenticate - so it gets one shared name (`SESSION_KEY`) and one shared vault mapping. An **encryption key** (Laravel's `APP_KEY`, Semaphore's access-key encryption - used to encrypt stored data) must never be shared, since compromising or rotating it can make one app's already-stored ciphertext unrecoverable, and that risk shouldn't leak to a second app sharing the same key. Give each app's encryption key the same semantic app-side name (`ENCRYPTION_KEY`) so its role is still obvious from the compose file, but map it to a distinct, app-prefixed GitHub Secret per app (`TWOFAUTH_ENCRYPTION_KEY`, `SEMAPHORE_ENCRYPTION_KEY`) so the values themselves are never shared.

Common shared variables a target's vaults map into one or more apps' `.env`:

- `DOMAIN` - Base domain for all services
- `CERTIFICATE_RESOLVER` - SSL resolver (Cloudflare DNS or HTTP challenge)
- `DATABASE_PASSWORD` - Shared database password
- `SESSION_KEY` - Shared session-signing key (safe to share - see above)
- `TIMEZONE` - System timezone
- `TELEGRAM_TOKEN`, `TELEGRAM_CHAT` - Shared Telegram bot for app-originated alerts

### Vault-Sourced vs. Internal Environment

Naming convention:
An app's `environment:` mix two different kinds of values: ones that must come from its vault (secrets, domain, feature flags - anything a deploy actually configures) and ones that are fixed or purely derived from `${APP_NAME}` (service hostnames, ports, internal db/user names - values that never change across deploys). Keeping them in one flat `x-environment` block makes it impossible to tell, at a glance, what an app actually needs from its vault.

When an app has both kinds, split into two anchors instead of one, declared in this order:

```yaml
x-vault-env: &vault-env
BASE_URL: https://${APP_NAME}.${DOMAIN}
BETTER_AUTH_SECRET: ${SESSION_KEY}
x-internal-env: &internal-env
NODE_ENV: production
POSTGRES_HOST: postgres
services:
backend:
environment:
<<: [*vault-env, *internal-env]
```

- `{APPNAME}_{VAR}` — app-specific variable where `{APPNAME}` is the uppercased app directory with hyphens replaced by underscores (e.g. `JELLYFIN_SHOWS_PATH`, `BESZEL_AGENT_PUBLIC_KEY`, `TWOFAUTH_APPS_DOMAIN`)
- `APPS_{VAR}` — server-wide variable shared across apps (e.g. `APPS_DOMAIN`, `APPS_TIMEZONE`)
Classification rule: a variable goes in `x-vault-env` if its value references *any* vault-sourced variable, even mixed with `${APP_NAME}` or literal text (e.g. `https://${APP_NAME}.${DOMAIN}` is vault-env, because it can't resolve to anything meaningful without `DOMAIN`). It goes in `x-internal-env` if its value is a hardcoded literal or derived only from `${APP_NAME}` - it would be exactly the same regardless of which vault fed the deploy.

The compose file is the source of truth for which overrides are allowed. Not every variable needs an app-specific override — only declare one when you actually want to allow it.
If an app's env is entirely one kind, use a single anchor named for that kind (`x-vault-env` or `x-internal-env`) instead of the generic `x-environment` - don't split into two just to leave one empty. No section comments above the anchors - the names are meant to be self-explanatory.

Common `APPS_*` variables a target's vaults map into one or more apps' `.env`:
`docker compose`'s multi-anchor merge key (`<<: [*a, *b]`) is what makes this work; verified it resolves identically to a single flat block via `docker compose config`.

- `APPS_DOMAIN` - Base domain for all services
- `APPS_CERTIFICATE_RESOLVER` - SSL resolver (Cloudflare DNS or HTTP challenge)
- `APPS_DATABASE_PASSWORD` - Shared database password
- `APPS_KEY_HEX_{16,32,64}` - Encryption keys for various apps
- `APPS_TIMEZONE` - System timezone
The same split applies to non-`environment` fields too, whenever an app has vault-configurable values there - e.g. `apps/beszel-agent/docker-compose.yml`'s `devices:` list is vault-sourced (`DISK_1_DEVICE`/`DISK_2_DEVICE`, each defaulting to `/dev/null` when unset), so it's declared as `x-vault-devices: &vault-devices` and referenced with `devices: *vault-devices`. A YAML anchor isn't limited to a map - it can hold a list just as well. Name the anchor for the compose field it targets (`x-vault-{field}`) so it's still obvious at a glance which fields the app's vault interface actually touches.

### Traefik Integration

All apps use Traefik labels pattern:

```yaml
traefik.enable=true
traefik.http.routers.${APP_NAME}.rule=Host(`${APP_NAME}.${APPS_DOMAIN}`)
traefik.http.routers.${APP_NAME}.tls.certresolver=${APPS_CERTIFICATE_RESOLVER}
traefik.http.routers.${APP_NAME}.rule=Host(`${APP_NAME}.${DOMAIN}`)
traefik.http.routers.${APP_NAME}.tls.certresolver=${CERTIFICATE_RESOLVER}
traefik.http.services.${APP_NAME}.loadbalancer.server.port=8080
```

Apps are accessible at `{app-name}.{APPS_DOMAIN}` with automatic SSL.
Apps are accessible at `{app-name}.{DOMAIN}` with automatic SSL.

## Operations

Expand Down Expand Up @@ -230,7 +248,7 @@ x-image: &image
# 3. X-ENVIRONMENT (only if environment variables exist)
x-environment: &environment
VAR1: ${VALUE1}
DATABASE_URL: postgresql://postgres:${APPS_DATABASE_PASSWORD}@postgres:5432/${APP_NAME}
DATABASE_URL: postgresql://postgres:${DATABASE_PASSWORD}@postgres:5432/${APP_NAME}

# 4. X-VOLUMES (if multiple services share volumes)
x-volumes: &volumes
Expand Down Expand Up @@ -258,7 +276,7 @@ services:
command: ["start", "--config", "/config.yml"]

# 4. USER (if required)
user: "${APPS_UID}:${APPS_GID}"
user: "${PUID}:${PGID}" # not UID/GID - those are shell-reserved

# 5. ENVIRONMENT (mandatory, via anchor)
environment: *environment
Expand Down Expand Up @@ -293,11 +311,11 @@ services:
### Key ordering principles:

1. **Include order**: networks.yml → database/cache templates (postgres/redis/mongodb/etc., pick a version) → others
2. **X-fields order**: x-image → x-environment → x-volumes (only if needed)
2. **X-fields order**: x-image → all `x-vault-*` anchors together → all `x-internal-*` anchors together → x-volumes (only if needed). Vault-sourced anchors are grouped first regardless of which compose field they target (e.g. `x-vault-env` then `x-vault-devices`, both before `x-internal-env`) - see "Vault-Sourced vs. Internal Environment" above
3. **X-image for shared images** - if multiple services use the same image, use `x-image: &image`
4. **X-volumes for shared volumes** - if 2+ volumes repeat across services, extract them to `x-volumes: &volumes` and merge with unique ones
5. **Image before extends** - declare what image is used, then extend common config
6. **Environment via anchor** - always use `x-environment: &environment` pattern
6. **Environment via anchor** - always declare env vars in an anchor, referenced with `environment: *name` (or merged via `<<: [*vault-env, *internal-env]` when split) - never inline a service's `environment:` map directly
7. **Networks from extends** - `main`, `main-http`, and `api` profiles include `traefik` and `internal`; never add `databases` (it's only for DB admin tools)
8. **Depends_on as simple list** - use array format without `condition:`, healthchecks are in common.yml
9. **Depends_on order**: postgres → redis → mongo → app services
Expand All @@ -324,7 +342,7 @@ include:
- ../networks.yml
- ../postgres-18.yml
x-environment: &environment
DATABASE_URL: postgresql://postgres:${APPS_DATABASE_PASSWORD}@postgres:5432/${APP_NAME}
DATABASE_URL: postgresql://postgres:${DATABASE_PASSWORD}@postgres:5432/${APP_NAME}
ENABLE_FEATURE: true
PORT: 8080
services:
Expand All @@ -334,7 +352,7 @@ services:
file: ../common.yml
service: main
command: ["worker", "--concurrency", "10"]
user: "${APPS_UID}:${APPS_GID}"
user: "${PUID}:${PGID}" # not UID/GID - those are shell-reserved
environment: *environment
volumes:
- ../../apps-data/${APP_NAME}/data:/data
Expand Down
15 changes: 4 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,16 +97,9 @@ This catalog is itself published as its own release asset (`flightdeck-apps.zip`

### Environment Variables

Every app's env comes from its own vault(s), declared in that target's manifest (see "Vaults And Targets" below). A vault declares the exact final variable names an app receives, mapped to GitHub Secret/Variable names - there is no server-side prefix filtering or shared root env file. Two apps' vaults can share a source secret (e.g. both mapping `APPS_DOMAIN`) without conflict, since each app ends up with its own separate `.env`.
Every app's env comes from its own vault(s), declared in that target's manifest (see "Vaults And Targets" below). A vault declares the exact final variable names an app receives, mapped to GitHub Secret/Variable names - there is no server-side prefix filtering or shared root env file. Two apps' vaults can share a source secret (e.g. both mapping `DOMAIN`) without conflict, since each app ends up with its own separate `.env`.

**Per-app overrides** are declared directly in each app's `docker-compose.yml` using bash fallback syntax:

```yaml
# App-specific override, falls back to server-wide value
SOME_PATH: ${MYAPP_SOME_PATH:-${APPS_SOME_PATH}}
```

To use the override for a given deploy, that app's own vault sets `MYAPP_SOME_PATH` in its `env:` mapping; to fall back to the shared value, the vault just omits it and relies on `APPS_SOME_PATH` alone. App prefixes are the uppercased app directory with hyphens replaced by underscores. See `AGENTS.md` for the full naming convention.
Variable names inside a compose file are always bare, never prefixed with the app's own name - each compose file is already scoped to one app. Whether a name is "shared" or app-specific only matters on the vault side (whether more than one app's vault maps it). See `AGENTS.md` for the full naming convention.

### Network Architecture

Expand Down Expand Up @@ -214,7 +207,7 @@ asset: mainframe-rybbit.sops.env
keys:
- mainframe
env:
APPS_DOMAIN: MAINFRAME_DOMAIN
DOMAIN: MAINFRAME_DOMAIN
```

`targets/mainframe.yml`:
Expand Down Expand Up @@ -243,7 +236,7 @@ credentials:
sops_age_key: MAINFRAME_AGE_PRIVATE_KEY
```

Credential fields contain GitHub Variable/Secret names, never credential values. `app_refs` and `hosts` are YAML arrays; `apps` is a mapping from app name to that app's own `env_refs` array. Each host uses the SSH `user@host` format. `app_refs` must list at least one app bundle — flightdeck's own `apps/` catalog is just another entry, not implicit. Each app in `apps` must list at least one `env_refs` entry; `deploy/deploy.py` decrypts and concatenates all of an app's sources into that app's own `.env` on the runner, failing loud on any key collision — but only within that one app's own sources. Two different apps' vaults sharing a key (e.g. both declaring `APPS_DOMAIN`) is expected, since each app gets a separate `.env`. `credentials.secrets.sops_age_key` names the GitHub Secret holding this target's *private* age key — the one used to decrypt its vaults, matching the public key in `keys/<target>.pub` used to encrypt them.
Credential fields contain GitHub Variable/Secret names, never credential values. `app_refs` and `hosts` are YAML arrays; `apps` is a mapping from app name to that app's own `env_refs` array. Each host uses the SSH `user@host` format. `app_refs` must list at least one app bundle — flightdeck's own `apps/` catalog is just another entry, not implicit. Each app in `apps` must list at least one `env_refs` entry; `deploy/deploy.py` decrypts and concatenates all of an app's sources into that app's own `.env` on the runner, failing loud on any key collision — but only within that one app's own sources. Two different apps' vaults sharing a key (e.g. both declaring `DOMAIN`) is expected, since each app gets a separate `.env`. `credentials.secrets.sops_age_key` names the GitHub Secret holding this target's *private* age key — the one used to decrypt its vaults, matching the public key in `keys/<target>.pub` used to encrypt them.

`load-yaml-matrix` reads every file in `vaults/` or `targets/` into a matrix — it does not validate the manifest shape. Each manifest's fields are the responsibility of whatever consumes them: `encrypt-env` re-parses and validates its own manifest from `manifest`, and the workflows calling `deploy-shared.yml` apply `path`/`keep-releases` defaults and pull `credentials.secrets`/`credentials.variables` values directly from the matrix item.

Expand Down
16 changes: 9 additions & 7 deletions apps/beszel-agent/docker-compose.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
x-environment: &environment
x-vault-env: &vault-env
KEY: ${PUBLIC_KEY}
x-vault-devices: &vault-devices
- ${DISK_1_DEVICE:-/dev/null}:${DISK_1_DEVICE:-/dev/null}
- ${DISK_2_DEVICE:-/dev/null}:${DISK_2_DEVICE:-/dev/null}
x-internal-env: &internal-env
PORT: 45876
HEALTHCHECK_CMD: "nc -z 127.0.0.1 $${PORT} || exit 1"
KEY: ${BESZEL_AGENT_PUBLIC_KEY}
services:
beszel-agent:
image: "henrygd/beszel-agent:alpine"
Expand All @@ -11,13 +14,12 @@ services:
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
- /home:/extra-filesystems/home:ro
environment: *environment
environment:
<<: [*vault-env, *internal-env]
cap_add:
- SYS_RAWIO # required for S.M.A.R.T. data
- SYS_ADMIN # required for NVMe S.M.A.R.T. data
devices:
- ${BESZEL_AGENT_DISK_1_DEVICE:-/dev/null}:${BESZEL_AGENT_DISK_1_DEVICE:-/dev/null}
- ${BESZEL_AGENT_DISK_2_DEVICE:-/dev/null}:${BESZEL_AGENT_DISK_2_DEVICE:-/dev/null}
devices: *vault-devices
healthcheck:
test: ["CMD", "/agent", "health"]
start_period: 30s
Expand Down
2 changes: 1 addition & 1 deletion apps/clickhouse-26.5.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ services:
environment:
CLICKHOUSE_DB: ${APP_NAME}
CLICKHOUSE_USER: ${APP_NAME}
CLICKHOUSE_PASSWORD: ${APPS_DATABASE_PASSWORD}
CLICKHOUSE_PASSWORD: ${DATABASE_PASSWORD}
CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT: 1
volumes:
- ../apps-data/${APP_NAME}/clickhouse/data:/var/lib/clickhouse
Expand Down
32 changes: 16 additions & 16 deletions apps/codecov/codecov.yml.tpl
Original file line number Diff line number Diff line change
@@ -1,37 +1,37 @@
setup:
# Replace with the http location of your Codecov
# https://docs.codecov.io/docs/configuration#section-codecov-url
codecov_url: https://${APP_NAME}.${CODECOV_APPS_DOMAIN:-${APPS_DOMAIN}}
codecov_url: https://${APP_NAME}.${DOMAIN}
# codecov_api_url: <codecov-url> # this defaults to <codecov-url> and is designed to work out of the box like this
# api_allowed_hosts: [] # this defaults to <codecov-url> and is designed to work out of the box like this
# Replace with your Codecov Enterprise License key. This is required for the containers to function.
# https://docs.codecov.io/docs/configuration#section-enterprise-license
enterprise_license: ${CODECOV_LICENSE}
enterprise_license: ${LICENSE}
admins: # https://docs.codecov.com/docs/configuration#instance-wide-admins
- service: github
username: ${CODECOV_ADMIN_GITHUB_USERNAME}
username: ${ADMIN_GITHUB_USERNAME}
http:
cookie_secret: ${APPS_KEY_HEX_32} # Replace it with a random string
cookies_domain: ${APP_NAME}.${CODECOV_APPS_DOMAIN:-${APPS_DOMAIN}}
cookie_secret: ${SESSION_KEY} # Replace it with a random string
cookies_domain: ${APP_NAME}.${DOMAIN}
timeseries:
enabled: true
guest_access: "off"
github:
client_id: ${CODECOV_GITHUB_CLIENT_ID}
client_secret: ${CODECOV_GITHUB_CLIENT_SECRET}
webhook_secret: ${CODECOV_GITHUB_WEBHOOK_SECRET}
client_id: ${GITHUB_CLIENT_ID}
client_secret: ${GITHUB_CLIENT_SECRET}
webhook_secret: ${GITHUB_WEBHOOK_SECRET}
integration:
id: ${CODECOV_GITHUB_APP_ID}
id: ${GITHUB_APP_ID}
pem: /config/key.pem
services:
redis_url: "redis://redis:6379"
database_url: "postgres://${APP_NAME}:${APPS_DATABASE_PASSWORD}@postgres:5432/${APP_NAME}"
timeseries_database_url: "postgres://${APP_NAME}:${APPS_DATABASE_PASSWORD}@timescale:5432/${APP_NAME}"
database_url: "postgres://${APP_NAME}:${DATABASE_PASSWORD}@postgres:5432/${APP_NAME}"
timeseries_database_url: "postgres://${APP_NAME}:${DATABASE_PASSWORD}@timescale:5432/${APP_NAME}"
minio:
host: ${APPS_S3_HOST}
bucket: ${CODECOV_S3_BUCKET}
region: ${APPS_S3_REGION}
host: ${S3_HOST}
bucket: ${S3_BUCKET}
region: ${S3_REGION}
verify_ssl: true
port: 443
access_key_id: ${APPS_S3_ACCESS_KEY_ID}
secret_access_key: ${APPS_S3_SECRET_ACCESS_KEY}
access_key_id: ${S3_ACCESS_KEY_ID}
secret_access_key: ${S3_SECRET_ACCESS_KEY}
Loading