nfrastack · cfoellmann · Aug 30, 2026
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,3 +1,18 @@
+## 4.10.0 2026-08-30 <foellmann at wus-technik dot com>
+
+   ### Added
+      - Multiple S3 targets per backup job via `S3_TARGETS`, uploading each backup to every
+        configured endpoint with identical content
+      - Per target configuration as `S3_<TARGET>_<OPTION>` / `DB01_S3_<TARGET>_<OPTION>`, falling
+        back to the existing single target `S3_` variables for everything a target does not override
+      - `_FILE` secret support for the per target variables
+
+   ### Changed
+      - S3 cleanup now sets up its own aws-cli environment per target instead of relying on the
+        values left behind by the preceding upload
+      - A failed S3 upload names the target it failed on and no longer aborts the remaining targets
+
+
 ## 4.9.2 2026-08-24 <code at nfrastack dot com>
 
    ### Changed

diff --git a/README.md b/README.md
@@ -333,6 +333,7 @@ If `DEFAULT_BACKUP_LOCATION` = `S3` then the following options are used:
 
 | Parameter                     | Description                                                                               | Default | `_FILE` |
 | ----------------------------- | ----------------------------------------------------------------------------------------- | ------- | ------- |
+| `DEFAULT_S3_TARGETS`          | Comma separated list of S3 targets to upload each backup to, e.g. `hetzner,impossible`    |         |         |
 | `DEFAULT_S3_BUCKET`           | S3 Bucket name e.g. `mybucket`                                                            |         | x       |
 | `DEFAULT_S3_KEY_ID`           | S3 Key ID (Optional)                                                                      |         | x       |
 | `DEFAULT_S3_KEY_SECRET`       | S3 Key Secret (Optional)                                                                  |         | x       |
@@ -347,6 +348,51 @@ If `DEFAULT_BACKUP_LOCATION` = `S3` then the following options are used:
 
 - When `DEFAULT_S3_KEY_ID` and/or `DEFAULT_S3_KEY_SECRET` is not set, will try to use IAM role assigned (if any) for uploading the backup files to S3 bucket.
 
+**Multiple S3 targets**
+
+Set `DEFAULT_S3_TARGETS` (or `DB01_S3_TARGETS` for a single job) to a comma separated list of
+names to upload every backup to more than one S3 endpoint. All targets receive the very same
+file, so the endpoints hold identical content:
+
+```yaml
+DEFAULT_S3_TARGETS: hetzner,impossible
+DEFAULT_S3_PATH: db                                      ## shared by both targets
+
+S3_HETZNER_HOST: fsn1.your-objectstorage.com
+S3_HETZNER_BUCKET: backups
+S3_HETZNER_KEY_ID: xxxxxxxx
+S3_HETZNER_KEY_SECRET: xxxxxxxx
+S3_HETZNER_REGION: eu-central
+
+S3_IMPOSSIBLE_HOST: eu-central-2.storage.impossibleapi.net
+S3_IMPOSSIBLE_BUCKET: backups-offsite
+S3_IMPOSSIBLE_KEY_ID: yyyyyyyy
+S3_IMPOSSIBLE_KEY_SECRET: yyyyyyyy
+S3_IMPOSSIBLE_REGION: eu-central-2
+```
+
+- Every S3 option above also exists per target as `S3_<TARGET>_<OPTION>` and per job as
+  `DB01_S3_<TARGET>_<OPTION>`. The value used for a target is the first one that is set of:
+
+  `DB01_S3_<TARGET>_<OPTION>` → `S3_<TARGET>_<OPTION>` → `DB01_S3_<OPTION>` → `S3_<OPTION>` → `DEFAULT_S3_<OPTION>`
+
+  Options that all targets share are therefore written once as `DEFAULT_S3_<OPTION>`, and only
+  what actually differs is repeated per target.
+- Target names are case insensitive, and any character that is not a letter or a digit becomes an
+  underscore - a target named `impossible-cloud` is configured through `S3_IMPOSSIBLE_CLOUD_*`.
+- `_FILE` secrets work for the per target variables too, e.g. `S3_HETZNER_KEY_SECRET_FILE`.
+- Every target is attempted, including after an earlier one failed, so a broken endpoint does not
+  stop the healthy ones from receiving their backup. A failed target is logged as an error naming
+  that target, raises the usual `Moving of backup ... reported errors` notification, and is handed
+  to post scripts as `$11` (`MOVE_EXIT_CODE`).
+- An unreachable endpoint is retried by `aws-cli` before it gives up, which can hold up the job for
+  several minutes. Shorten this per target with e.g.
+  `S3_HETZNER_EXTRA_OPTS: --cli-connect-timeout 10 --cli-read-timeout 30`.
+- `DEFAULT_CLEANUP_TIME` applies to all targets alike. Cleanup is skipped for a job that ran into
+  errors, so a failing target also postpones the cleanup on the healthy ones.
+- Leaving `S3_TARGETS` unset keeps the previous single target behaviour of the `DEFAULT_S3_*` and
+  `DB01_S3_*` variables unchanged.
+
 ###### Azure
 
 If `DEFAULT_BACKUP_LOCATION` = `blobxfer` then the following options are used:.
@@ -619,6 +665,7 @@ If `DB01_BACKUP_LOCATION` = `S3` then the following options are used:
 
 | Parameter                  | Description                                                                               | Default | `_FILE` |
 | -------------------------- | ----------------------------------------------------------------------------------------- | ------- | ------- |
+| `DB01_S3_TARGETS`          | Comma separated list of S3 targets to upload each backup to, e.g. `hetzner,impossible`    |         |         |
 | `DB01_S3_BUCKET`           | S3 Bucket name e.g. `mybucket`                                                            |         | x       |
 | `DB01_S3_KEY_ID`           | S3 Key ID (Optional)                                                                      |         | x       |
 | `DB01_S3_KEY_SECRET`       | S3 Key Secret (Optional)                                                                  |         | x       |
@@ -633,6 +680,12 @@ If `DB01_BACKUP_LOCATION` = `S3` then the following options are used:
 
 > When `DB01_S3_KEY_ID` and/or `DB01_S3_KEY_SECRET` is not set, will try to use IAM role assigned (if any) for uploading the backup files to S3 bucket.
 
+**Multiple S3 targets**
+
+Set `DB01_S3_TARGETS` to a comma separated list of names to upload this job's backups to more than
+one S3 endpoint, each target configured through `DB01_S3_<TARGET>_<OPTION>`. See
+[Multiple S3 targets](#s3) under the job defaults for the lookup order and the behaviour on failure.
+
 ###### Azure
 
 If `DB01_BACKUP_LOCATION` = `blobxfer` then the following options are used:.

diff --git a/examples/multiple-s3-targets/compose.yml b/examples/multiple-s3-targets/compose.yml
@@ -0,0 +1,76 @@
+#
+# Example for uploading every backup to two S3 endpoints at once
+#
+# Both endpoints receive the identical dump and checksum file. Options that both targets
+# share are written once as DEFAULT_S3_*, only what differs is repeated per target.
+#
+
+services:
+  example-multi-s3-db:
+    hostname: example-db-host
+    image: docker.io/library/mariadb:11
+    container_name: example-multi-s3-db
+    restart: unless-stopped
+    networks:
+      example-multi-s3-net:
+    environment:
+      MARIADB_ROOT_PASSWORD: rootpass
+      MARIADB_DATABASE: example
+
+  # First endpoint - stands in for any S3 compatible service
+  example-multi-s3-minio:
+    hostname: minio
+    image: quay.io/minio/minio:latest
+    container_name: example-multi-s3-minio
+    restart: unless-stopped
+    command: server /data
+    networks:
+      example-multi-s3-net:
+    environment:
+      MINIO_ROOT_USER: miniokey
+      MINIO_ROOT_PASSWORD: miniosecret
+
+  example-multi-s3-db-backup:
+    container_name: example-multi-s3-db-backup
+    image: nfrastack/db-backup
+    restart: always
+    networks:
+      example-multi-s3-net:
+    environment:
+      - TIMEZONE=Europe/Berlin
+      - CONTAINER_ENABLE_MONITORING=FALSE
+      - CONTAINER_NAME=example-multi-s3-db-backup
+      # - DEBUG_MODE=TRUE
+
+      - DB01_TYPE=mysql
+      - DB01_HOST=example-db-host
+      - DB01_NAME=example
+      - DB01_USER=root
+      - DB01_PASS=rootpass
+      - DB01_BACKUP_INTERVAL=1           # backup every minute
+      - DB01_CLEANUP_TIME=60             # applies to every target alike
+
+      - DEFAULT_BACKUP_LOCATION=S3
+      - DEFAULT_S3_PATH=example          # shared by both targets
+
+      # Upload to both targets, in this order
+      - DEFAULT_S3_TARGETS=local,offsite
+
+      - S3_LOCAL_HOST=minio:9000
+      - S3_LOCAL_PROTOCOL=http
+      - S3_LOCAL_BUCKET=backups
+      - S3_LOCAL_KEY_ID=miniokey
+      - S3_LOCAL_KEY_SECRET=miniosecret
+      - S3_LOCAL_REGION=us-east-1
+
+      - S3_OFFSITE_HOST=s3.example.com
+      - S3_OFFSITE_BUCKET=backups-offsite
+      - S3_OFFSITE_KEY_ID=xxxxxxxx
+      - S3_OFFSITE_KEY_SECRET=xxxxxxxx
+      - S3_OFFSITE_REGION=eu-central-1
+      # Do not let an unreachable offsite endpoint stall the job for minutes
+      - S3_OFFSITE_EXTRA_OPTS=--cli-connect-timeout 10 --cli-read-timeout 30
+
+networks:
+  example-multi-s3-net:
+    name: example-multi-s3-net