Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 0 additions & 7 deletions docker.mk
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,6 @@ NAME ?= default
DKR_COMPOSE_FILE ?= -f $(PWD)/docker-compose.yaml
DKR_COMPOSE_PROJECT = $(NAME)

# path to the source tree bind-mounted into the containers; defaults to the main
# checkout. override to run against a git worktree instead (see php_tests_worktree
# in php.mk). exported so `docker compose` can interpolate ${DKR_COMPOSE_SRC} in
# your compose file's volume definitions.
DKR_COMPOSE_SRC ?= $(PWD)
export DKR_COMPOSE_SRC

ifneq ($(shell docker compose --version),)
DKR_COMPOSE_CMD = COMPOSE_PROFILES=$(DKR_COMPOSE_PROFILES) docker compose $(DKR_COMPOSE_FILE) -p $(DKR_COMPOSE_PROJECT)
else
Expand Down
20 changes: 0 additions & 20 deletions docs/docker.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@
* [setup](#setup)
* [variables](#variables)
* [docker compose file](#docker-compose-file)
* [source path](#source-path)
* [commands](#commands)
* [dkr_pull](#dkr_pull)
* [dkr_build](#dkr_build)
Expand All @@ -30,25 +29,6 @@ The variable `$(GIT_TAG)` comes from [common.mk](../common.mk) and documented in
dkr_build: DKR_COMPOSE_FILE = -f ./docker-compose.build.yaml
```

### source path

`DKR_COMPOSE_SRC` is the path to the source tree bind-mounted into the containers.
It defaults to `$(PWD)` (the main checkout) and is exported so `docker compose` can
interpolate it inside your compose file:

```yaml
services:
cli:
# no `container_name:` — lets an isolated run coexist with the main stack
volumes:
- ${DKR_COMPOSE_SRC:-.}:/app
```

Referencing `${DKR_COMPOSE_SRC}` instead of a hardcoded path, and omitting
`container_name`, is what lets [`php_tests_worktree`](./php.md#tests-against-a-worktree)
run a git worktree's code against the already-running main-checkout services without
conflicting with them.

## commands

### dkr_pull
Expand Down
41 changes: 24 additions & 17 deletions docs/php.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,11 +53,17 @@ make php_tests_behat

### tests against a worktree

Run the suite against a git worktree's code while reusing the already-running local
services (MySQL / Redis / Elasticsearch), with per-worktree database isolation so
parallel worktree runs don't collide with each other or the main checkout:
Run the suite against a git [worktree's](https://git-scm.com/docs/git-worktree) code while
reusing the already-running local stack (image, compose files, secrets, services), with a
per-worktree database so parallel worktree runs never collide with each other or the main
checkout.

This needs **no changes in the consuming repo**. Bring the main stack up as usual and run
this from the MAIN checkout, pointing at a worktree — decrypted env/secret files (which are
gitignored and absent in a worktree) are still sourced from the main stack:

```shell
make dkr_up_local # once: the stack it reuses must be up
make php_tests_worktree WORKTREE=/path/to/worktree
```

Expand All @@ -70,17 +76,18 @@ make php_tests_worktree WORKTREE=/path/to/worktree COMMAND="php artisan test"

How it works:

* **source** — the worktree is bind-mounted via `DKR_COMPOSE_SRC` (see
[docker.md](./docker.md#source-path)) rather than the main checkout.
* **services** — the container runs with `--no-deps`, reusing the services you already
have `up` instead of starting a duplicate stack, so bring the main stack up first.
* **database isolation** — the run is given a unique, sanitised `DB_DATABASE` derived from
the worktree path (override with `DB_DATABASE=...`). The shared MySQL server is reused;
only the schema differs. Ensure your test bootstrap creates and migrates it (e.g.
Laravel's `RefreshDatabase`).
* **composer drift** — if the worktree's `composer.lock` differs from the main
checkout's (or its `vendor/` is missing) dependencies are installed for the worktree
first, otherwise the existing `vendor/` is reused.

This requires your compose file to reference `${DKR_COMPOSE_SRC}` for the code volume and
to avoid a hardcoded `container_name` — see [docker.md](./docker.md#source-path).
* **stack reuse** — it finds the running `cli` container for the compose project and reuses
the exact compose files, image and services that created it (so it never rebuilds a stale
image, and secrets come from the main stack's `env_file`). Bring the stack up first.
* **source** — only the code bind-mount is redirected to the worktree, via an auto-generated
ephemeral compose override — your compose files are not edited. The app directory
(the one holding `composer.json`) is auto-detected from the running mount; override with
`APP_DIR=<path relative to the repo root>` if detection can't find it.
* **database isolation** — the run gets a unique, sanitised `DB_DATABASE` derived from the
worktree path (override with `DB_DATABASE=...`), reusing the shared MySQL server. The
database is created, granted to the app user, and **migrated** before the suite runs.
Add `DB_SEED=1` for suites whose feature tests need seeded reference data, or `DB_FRESH=1`
to drop and rebuild it for a clean slate.
* **composer drift** — if the worktree's `composer.lock` differs from the main checkout's
(or its `vendor/` is missing) dependencies are installed into the worktree first;
otherwise the worktree's `vendor/` is reused, so re-runs are fast.
150 changes: 136 additions & 14 deletions php.mk
Original file line number Diff line number Diff line change
Expand Up @@ -14,31 +14,153 @@ php_composer_update:
php_tests_%:
$(DKR_COMPOSE_CMD_UP) || { $(DKR_COMPOSE_CMD_DOWN); exit 1; }

# php_tests_worktree — run the test suite against a git worktree's code while
# reusing the already-running local services (MySQL / Redis / Elasticsearch),
# with per-worktree database isolation so parallel worktree runs don't collide
# with each other or the main checkout. bring the main stack up first.
# php_tests_worktree — run a repo's test suite against a git worktree's code while
# reusing the already-running local stack (image, compose files, secrets, services),
# with a per-worktree database so parallel worktree runs never collide with each
# other or the main checkout.
#
# It needs ZERO changes in the consuming repo: bring the main stack up as usual
# (e.g. `make dkr_up_local`) and run this from the MAIN checkout, pointing at a
# worktree. It discovers the running cli container, reuses the exact compose files
# and image that created it, and only redirects the code bind-mount to the worktree
# via an auto-generated ephemeral override — main's env_file/secrets are untouched
# (which matters because decrypted env files are gitignored and absent in worktrees).
#
# make php_tests_worktree WORKTREE=/path/to/worktree
# make php_tests_worktree WORKTREE=/path/to/worktree TESTS=behat
# make php_tests_worktree WORKTREE=/path/to/worktree COMMAND="php artisan test"
php_tests_worktree: CONTAINER ?= cli
#
# Knobs (auto-detected unless set):
# APP_DIR path to the app (composer.json/vendor) relative to the repo root;
# auto-detected from the running code mount (e.g. api/laravel).
# DB_DATABASE the isolated database name; derived from the worktree path.
# DB_FRESH=1 drop the isolated database first for a clean slate.
# DB_SEED=1 also run `php artisan db:seed` after migrating, for suites whose
# feature tests expect seeded reference data.
define PHP_TESTS_WORKTREE
log() { printf 'php_tests_worktree: %s\n' "$$1" >&2; }
die() { log "$$1"; exit 1; }

if docker compose version >/dev/null 2>&1; then dc="docker compose"; else dc="docker-compose"; fi
tab=$$(printf '\t')
[ -n "$$CLI_SERVICE" ] || CLI_SERVICE=cli

[ -n "$$WORKTREE" ] && [ -d "$$WORKTREE" ] || die "WORKTREE '$$WORKTREE' does not exist"

# 1. find the running cli container for this compose project + service.
cli=$$(docker ps --filter "label=com.docker.compose.project=$$PROJECT" --filter "label=com.docker.compose.service=$$CLI_SERVICE" --format '{{.ID}}' | head -n1)
[ -n "$$cli" ] || die "no running '$$CLI_SERVICE' container in compose project '$$PROJECT' - bring the stack up first (e.g. make dkr_up_local)"

# 2. reuse the EXACT compose files and image the running stack was created from.
cfg=$$(docker inspect "$$cli" --format '{{index .Config.Labels "com.docker.compose.project.config_files"}}')
[ -n "$$cfg" ] || die "could not read the compose config files from the running stack"
files="-f $$(printf '%s' "$$cfg" | sed 's/,/ -f /g')"
image=$$(docker inspect "$$cli" --format '{{.Image}}')

# 3. locate the code mount and its container target. auto-detect it as the
# shallowest bind mount, INSIDE the checkout, whose source holds composer.json
# (so home mounts like ~/.composer or ~/.ssh can't be mistaken for the app).
mounts=$$(docker inspect "$$cli" --format '{{range .Mounts}}{{if eq .Type "bind"}}{{.Source}}{{"\t"}}{{.Destination}}{{"\n"}}{{end}}{{end}}')
if [ -n "$$APP_DIR" ]; then
# normalise: drop a leading ./ and any trailing slashes so '.', 'x/' etc. still
# match the canonical Docker mount source.
ad=$${APP_DIR#./}; while [ "$${ad%/}" != "$$ad" ]; do ad=$${ad%/}; done
if [ -z "$$ad" ] || [ "$$ad" = . ]; then app_host="$$MAIN"; else app_host="$$MAIN/$$ad"; fi
app_target=$$(printf '%s\n' "$$mounts" | awk -F"$$tab" -v s="$$app_host" '$$1==s{print $$2; exit}')
[ -n "$$app_target" ] || die "APP_DIR='$$APP_DIR' is not bind-mounted into the running '$$CLI_SERVICE' container"
Comment thread
coderabbitai[bot] marked this conversation as resolved.
else
line=$$(printf '%s\n' "$$mounts" | while IFS="$$tab" read -r src dst; do
[ -n "$$src" ] || continue
[ "$$src" = "$$MAIN" ] || [ "$${src#"$$MAIN"/}" != "$$src" ] || continue
[ -f "$$src/composer.json" ] || continue
depth=$$(printf '%s' "$$src" | tr -cd / | wc -c | tr -d ' ')
printf '%s%s%s%s%s\n' "$$depth" "$$tab" "$$src" "$$tab" "$$dst"
done | sort -n | head -n1 | cut -f2-)
app_host=$$(printf '%s' "$$line" | cut -f1)
app_target=$$(printf '%s' "$$line" | cut -f2)
fi
[ -n "$$app_host" ] && [ -n "$$app_target" ] || die "could not locate the app dir (no bind mount under '$$MAIN' with composer.json); pass APP_DIR=<path relative to the repo root>"

# same relative path inside the worktree.
case "$$app_host" in
"$$MAIN") app_rel='.' ;;
"$$MAIN"/*) app_rel=$${app_host#"$$MAIN"/} ;;
*) die "detected app dir '$$app_host' is outside the checkout '$$MAIN'; pass APP_DIR" ;;
esac
wt_app="$$WORKTREE/$$app_rel"
[ -d "$$wt_app" ] || die "the worktree has no app dir at '$$wt_app' (relative path '$$app_rel')"

log "worktree='$$WORKTREE' app='$$app_rel' target='$$app_target' db='$$DB'"
log "reusing the image and running services of compose project '$$PROJECT'"

# 4. ephemeral override: pin the running image + redirect ONLY the code mount to the
# worktree. compose merges volumes by target (override wins) so the consumer repo
# needs no compose edits; main's compose still provides env_file/secrets/network.
override=$$(mktemp -t php_tests_worktree.XXXXXX.yaml)
trap 'rm -f "$$override"' EXIT INT TERM HUP
printf 'services:\n %s:\n image: %s\n volumes:\n - %s:%s\n' "$$CLI_SERVICE" "$$image" "$$wt_app" "$$app_target" > "$$override"

run() { $$dc $$files -f "$$override" -p "$$PROJECT" run --rm --no-deps "$$@"; }

# 5. composer: install into the worktree when its lock drifts from main (or its
# vendor is missing); otherwise reuse the worktree's vendor for fast re-runs.
if [ ! -d "$$wt_app/vendor" ] || ! cmp -s "$$wt_app/composer.lock" "$$app_host/composer.lock"; then
log "composer.lock drift or missing vendor - installing dependencies for the worktree"
run -e ENTRYPOINT_SKIP=true "$$CLI_SERVICE" composer install --no-interaction
else
log "composer.lock matches the main checkout - reusing the worktree's vendor"
fi

# 6. provision the isolated database: create it + grant the app user, then migrate.
# the app user usually only has rights on the base schema, so create as root using
# the running mysql container's OWN root password env - keeping this zero-config.
dbc=$$(docker ps --filter "label=com.docker.compose.project=$$PROJECT" --format '{{.Names}}{{"\t"}}{{.Image}}' | awk -F"$$tab" 'tolower($$2) ~ /mysql|mariadb|percona/ {print $$1; exit}')
if [ -n "$$dbc" ]; then
appuser=$$(docker exec "$$cli" printenv DB_USERNAME 2>/dev/null || true)
[ -n "$$appuser" ] || appuser=root
if [ "$$DB_FRESH" = 1 ] || [ "$$DB_FRESH" = true ]; then
log "DB_FRESH set - dropping database '$$DB'"
docker exec -e ISO="$$DB" "$$dbc" sh -c 'mysql -uroot -p"$$MYSQL_ROOT_PASSWORD" -e "DROP DATABASE IF EXISTS \`$$ISO\`"' >/dev/null 2>&1 || die "could not drop database '$$DB' via '$$dbc'"
fi
Comment thread
coderabbitai[bot] marked this conversation as resolved.
log "ensuring isolated database '$$DB' exists and is granted to '$$appuser'"
docker exec -e ISO="$$DB" -e APPUSER="$$appuser" "$$dbc" sh -c 'mysql -uroot -p"$$MYSQL_ROOT_PASSWORD" -e "CREATE DATABASE IF NOT EXISTS \`$$ISO\`; GRANT ALL PRIVILEGES ON \`$$ISO\`.* TO \`$$APPUSER\`@\`%\`; FLUSH PRIVILEGES;"' >/dev/null 2>&1 || die "could not create/grant database '$$DB' via '$$dbc' (expects a standard mysql/mariadb image exposing MYSQL_ROOT_PASSWORD)"
prep="php artisan migrate --force"
{ [ "$$DB_SEED" = 1 ] || [ "$$DB_SEED" = true ]; } && prep="$$prep && php artisan db:seed --force" || true
log "preparing isolated database '$$DB' ($$prep)"
run -e ENTRYPOINT_SKIP=true --env DB_DATABASE="$$DB" "$$CLI_SERVICE" sh -c "$$prep"
else
log "WARNING: no mysql/mariadb/percona service found in project '$$PROJECT' - skipping database provisioning; DB-backed tests may fail"
fi

# 7. run the suite against the worktree's code, in the isolated database.
log "running: $$COMMAND"
if [ "$$ESKIP" = true ]; then
run -e ENTRYPOINT_SKIP=true --env DB_DATABASE="$$DB" "$$CLI_SERVICE" $$COMMAND
else
run --env DB_DATABASE="$$DB" "$$CLI_SERVICE" $$COMMAND
fi
endef
export PHP_TESTS_WORKTREE

php_tests_worktree: WORKTREE ?= $(PWD)
php_tests_worktree: WORKTREE_SRC = $(abspath $(WORKTREE))
php_tests_worktree: WORKTREE_ID = $(notdir $(WORKTREE_SRC))
php_tests_worktree: DKR_COMPOSE_SRC = $(WORKTREE_SRC)
# db name: sanitised worktree basename plus a hash of the full path, so two
# worktrees that share a basename under different parents don't collide and odd
# characters (dots, slashes) can't produce an invalid identifier.
# db name: sanitised worktree basename plus a hash of the full path, so two worktrees
# that share a basename under different parents don't collide and odd characters
# (dots, slashes) can't produce an invalid identifier.
php_tests_worktree: DB_DATABASE ?= $(shell printf 'test_%s_%s' "$$(printf '%s' '$(WORKTREE_ID)' | tr -c 'A-Za-z0-9' '_')" "$$(printf '%s' '$(WORKTREE_SRC)' | cksum | cut -d' ' -f1)")
php_tests_worktree: TESTS ?= phpunit
php_tests_worktree: COMMAND ?= vendor/bin/$(TESTS)
php_tests_worktree: DKR_COMPOSE_ADDITIONAL_RUN = --no-deps --env DB_DATABASE=$(DB_DATABASE)
php_tests_worktree: APP_DIR ?=
php_tests_worktree: DB_FRESH ?=
php_tests_worktree: DB_SEED ?=
php_tests_worktree: WORKTREE_ENTRYPOINT_SKIP ?= true
php_tests_worktree:
@test -n "$(WORKTREE)" && test -d "$(WORKTREE_SRC)" || { echo "php_tests_worktree: WORKTREE '$(WORKTREE)' does not exist" >&2; exit 1; }
@echo "php_tests_worktree: running '$(COMMAND)' against $(WORKTREE_SRC) (db: $(DB_DATABASE))"
@if [ ! -d "$(WORKTREE_SRC)/vendor" ] || ! cmp -s "$(WORKTREE_SRC)/composer.lock" "$(PWD)/composer.lock"; then echo "php_tests_worktree: composer.lock drift detected - installing dependencies for '$(WORKTREE_ID)'"; $(DKR_COMPOSE_CMD) run --rm $(DKR_COMPOSE_ADDITIONAL)--no-deps $(CONTAINER) composer install; else echo "php_tests_worktree: composer.lock matches main checkout - reusing vendor"; fi
$(DKR_COMPOSE_CMD_RUN)
@MAIN='$(PWD)' WORKTREE='$(WORKTREE_SRC)' PROJECT='$(DKR_COMPOSE_PROJECT)' \
CLI_SERVICE='$(CONTAINER)' DB='$(DB_DATABASE)' COMMAND='$(COMMAND)' \
APP_DIR='$(APP_DIR)' DB_FRESH='$(DB_FRESH)' DB_SEED='$(DB_SEED)' \
ESKIP='$(WORKTREE_ENTRYPOINT_SKIP)' \
sh -eu -c "$$PHP_TESTS_WORKTREE"

php_cmd_%: CONTAINER ?= cli
php_cmd_%:
Expand Down
Loading
Loading