diff --git a/CHANGELOG.md b/CHANGELOG.md index 4ba9cd9..c3f6e3d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -27,12 +27,30 @@ `test_command_rollback_tracking.py` — all using real Postgres databases (no mocks), covering idempotent table creation, hash normalization, conflict detection, empty-diff hops, safe mode, and rollback tracking. +- **Migration execution** (`--apply`): + - Executes the generated migration against `dburl_from` in a single + transaction (all-or-nothing — a failed statement rolls back everything + that ran before it) instead of only printing the SQL + - On success, automatically records the migration in `dburl_from`'s + `migradiff_history` table — no need to also pass `--record-history` + - On failure, nothing is recorded (since nothing was actually applied) + and the command exits with code 4 + - Not supported with `--from-file` (no live database to apply to) or + `--promote` (not implemented yet) — both are rejected with a clear + error before anything runs + - New helper `_apply_migration()` in `migra/command.py`; JSON output + (`--output json`) gets a new `"apply"` object with the outcome + - New tests: `test_command_apply.py` (11 tests, real Postgres, including + a direct atomicity test that forces a mid-migration failure) ### Notes - This is the foundational layer for the upcoming "Control Plane" feature set. -- Known limitation: `migradiff_history` records migrations as *generated*, - not *applied* — see the README "Migration State Tracking" section for details. +- Resolved: `migradiff_history` used to only record migrations as + *generated*, not *applied* — `--apply` closes that gap by executing the + migration itself and only recording history on confirmed success. Plain + `--record-history` (without `--apply`) still only means "generated", not + "applied" — see the README "Migration State Tracking" section. - All new features work without AI extras (`anthropic` optional dependency). ## [1.7.2] - 2026-06-08 diff --git a/CLAUDE.md b/CLAUDE.md index fe650af..a5c9697 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -64,15 +64,22 @@ branch, and publishes to PyPI + tags a release from `master`. (`_check_for_destructive`), column-rename detection (`detect_column_renames`, vs. a naive drop+add), risk classification for `--output json` (`classify_sql_statement`), credential redaction in error output (`redact_credentials`). -- `--status`/`--history`/`--promote`/`--record-rollback` etc. talk to `migra/history.py`. +- `--status`/`--history`/`--promote`/`--record-rollback`/`--apply` etc. talk to `migra/history.py`. ### Migration state tracking (`migra/history.py`) -A `migradiff_history` table (`HISTORY_TABLE`) recorded into the *target* database, keyed by a -SHA-256 hash of normalized SQL (`compute_migration_hash`). This is the basis for `--promote` -(multi-environment promotion) and `--record-rollback`. **Important semantic**: a history row means -a migration was generated/reviewed for that target, not that migra itself executed it — migra never -applies SQL to a live database. Preserve this distinction in any related code or docs. +A `migradiff_history` table (`HISTORY_TABLE`), keyed by a SHA-256 hash of normalized SQL +(`compute_migration_hash`). This is the basis for `--promote` (multi-environment promotion), +`--record-rollback`, and `--apply`. **Important semantic, easy to get backwards**: a plain +`--record-history` (without `--apply`) writes into `dburl_target`'s history table and only means +"this migration was generated/reviewed for that target" — not that it was executed anywhere. +`--apply` (`_apply_migration()` in `command.py`) is the one path that actually executes SQL: it runs +the migration against `dburl_from` (the database being migrated — see the CLI's own help text and +the README's `psql dburl_from < migration.sql` convention) inside a single transaction, and only on +confirmed success does it record history — into `dburl_from`'s table, not `dburl_target`'s. A failed +`--apply` rolls back everything and records nothing. `--apply` is rejected up front (before any +dispatch) when combined with `--from-file` (dburl_from would be an ephemeral throwaway database) or +`--promote` (the chain's from/to direction is not yet reconciled with `--apply`'s execution target). ### AI features (optional extra: `pip install migradiff[ai]`, needs `ANTHROPIC_API_KEY`) diff --git a/PROJECT_PLAN.md b/PROJECT_PLAN.md index eb695f6..8ef249d 100644 --- a/PROJECT_PLAN.md +++ b/PROJECT_PLAN.md @@ -100,24 +100,27 @@ done. See "Completed Features" above for what actually shipped: (fr, de, ja, zh, hi, he — 6 languages, merged as a docs-only PR, no dedicated version bump). -### Next: release migration state tracking as v1.8.0 +### Shipped: migration state tracking and `--apply` -`--status`/`--history`/`--promote`/`--record-rollback` (see "Completed -Features") is code-complete and fully tested but still sitting on branch -`new-feature-6-18-2026`. Landing this — merge, version bump, tag, PyPI -release — is the immediate next step, ahead of any new feature work below. +`--status`/`--history`/`--promote`/`--record-rollback` merged into `main` +2026-08-08 (PRs #8–#10; no version bump/tag yet — still on v1.7.2). +`--apply` (execute the migration against `dburl_from` instead of only +printing it, with automatic history recording on confirmed success — see +README's "Applying Migrations" section) shipped shortly after on top of +that. Neither has been cut into a tagged release yet; `pyproject.toml` +still reads 1.7.2. Bumping to v1.8.0 and publishing is still outstanding. ### Planned (Backlog) | Feature | Effort | Value | Notes | |---------|--------|-------|-------| -| `--apply` | Medium | High | Execute generated SQL directly instead of requiring the user to pipe to `psql`. Also needed so `migradiff_history` can record true "applied" state instead of just "generated/reviewed" — see the known limitation in README's Migration State Tracking section. | | Native `--fail-on-destructive` flag | Low | High | Currently this behavior only exists inside the GitHub Action's `action-entrypoint.sh`. CircleCI/GitLab/pre-commit users have no equivalent without wrapping the CLI themselves. | +| `--apply` + `--promote` | Medium | Medium | `--apply` currently refuses to run when `--promote` is also given — `--promote`'s from/to direction needs to be reconciled with which database `--apply` should actually execute against before this is safe to wire up. | | `--document` | Medium-High | High | Schema documentation generation | | Multi-schema hardening | Medium | High | `Migration.__init__` (`migra/migra.py`) still hard-rejects `schema` + `exclude_schema` together; cross-schema FK/dependency ordering in `add_all_changes()` hasn't had a dedicated multi-tenant test pass | | pgvector support | Low | Medium | Modern Postgres vector types — unconfirmed whether `schemainspect` already round-trips them | | `--suggest-indexes` | Medium | Medium | AI recommends useful indexes; can reuse `AIAdvisor`'s existing table-stats extraction | -| `--dry-run` | Low | Medium | Only meaningful once `--apply` exists | +| `--dry-run` | Low | Medium | Preview what `--apply` would do without executing it | --- @@ -317,21 +320,18 @@ without a dedicated version bump — not tied to a CHANGELOG entry. ## Next Steps -1. **Land migration state tracking as v1.8.0** - - Merge `new-feature-6-18-2026` — code-complete, 342/342 tests passing, - flake8/black clean, and CLI-smoke-tested end-to-end as of 2026-08-08 - - Bump version, tag release, publish to PyPI +1. **Cut v1.8.0** — migration state tracking and `--apply` are both merged + to `main` and fully tested, but `pyproject.toml` is still at 1.7.2 and + nothing has been tagged/published to PyPI yet. -2. **`--apply` flag** - - Closes the "generated vs. applied" gap called out in README's - Migration State Tracking known limitation - - Should auto-call the equivalent of `--record-history` on success - -3. **Native `--fail-on-destructive` CLI flag** +2. **Native `--fail-on-destructive` CLI flag** - Promote the GitHub Action's destructive-detection behavior into `command.py` itself so non-Action CI users (CircleCI, GitLab, plain scripts) get it too +3. **Reconcile `--promote`'s direction with `--apply`** before wiring the + two together — see the Planned/Backlog table above. + 4. **Post v1.8.0:** Enterprise tier planning - Design licensing system - Plan hosted features @@ -339,6 +339,6 @@ without a dedicated version bump — not tied to a CHANGELOG entry. --- -**Document version:** Updated 2026-08-08 (reconciled with actual repo/code state) +**Document version:** Updated 2026-08-08 (reconciled with actual repo/code state, incl. `--apply`) **Last updated by:** Claude (with Leo) **Repository:** https://github.com/postgresql-tools/migra diff --git a/PROJECT_PLAN2.md b/PROJECT_PLAN2.md index 3fccbcc..e37a31a 100644 --- a/PROJECT_PLAN2.md +++ b/PROJECT_PLAN2.md @@ -103,24 +103,27 @@ Hindi/Hebrew for the language set — that changed during implementation. The 6 languages that actually shipped are fr, de, ja, zh, hi, he (no Spanish); see `PROJECT_PLAN.md` for the corrected rationale. -### Next: release migration state tracking as v1.8.0 +### Shipped: migration state tracking and `--apply` -`--status`/`--history`/`--promote`/`--record-rollback` (see "Completed -Features") is code-complete and fully tested but still sitting on branch -`new-feature-6-18-2026`. Landing this — merge, version bump, tag, PyPI -release — is the immediate next step, ahead of any new feature work below. +`--status`/`--history`/`--promote`/`--record-rollback` merged into `main` +2026-08-08 (PRs #8–#10; no version bump/tag yet — still on v1.7.2). +`--apply` (execute the migration against `dburl_from` instead of only +printing it, with automatic history recording on confirmed success — see +README's "Applying Migrations" section) shipped shortly after on top of +that. Neither has been cut into a tagged release yet; `pyproject.toml` +still reads 1.7.2. Bumping to v1.8.0 and publishing is still outstanding. ### Planned (Backlog) | Feature | Effort | Value | Notes | |---------|--------|-------|-------| -| `--apply` | Medium | High | Execute generated SQL directly instead of requiring the user to pipe to `psql`. Also needed so `migradiff_history` can record true "applied" state instead of just "generated/reviewed" — see the known limitation in README's Migration State Tracking section. | | Native `--fail-on-destructive` flag | Low | High | Currently this behavior only exists inside the GitHub Action's `action-entrypoint.sh`. CircleCI/GitLab/pre-commit users have no equivalent without wrapping the CLI themselves. | +| `--apply` + `--promote` | Medium | Medium | `--apply` currently refuses to run when `--promote` is also given — `--promote`'s from/to direction needs to be reconciled with which database `--apply` should actually execute against before this is safe to wire up. | | `--document` | Medium-High | High | Schema documentation generation | | Multi-schema hardening | Medium | High | `Migration.__init__` (`migra/migra.py`) still hard-rejects `schema` + `exclude_schema` together; cross-schema FK/dependency ordering in `add_all_changes()` hasn't had a dedicated multi-tenant test pass | | pgvector support | Low | Medium | Modern Postgres vector types — unconfirmed whether `schemainspect` already round-trips them | | `--suggest-indexes` | Medium | Medium | AI recommends useful indexes; can reuse `AIAdvisor`'s existing table-stats extraction | -| `--dry-run` | Low | Medium | Only meaningful once `--apply` exists | +| `--dry-run` | Low | Medium | Preview what `--apply` would do without executing it | --- @@ -320,21 +323,18 @@ without a dedicated version bump — not tied to a CHANGELOG entry. ## Next Steps -1. **Land migration state tracking as v1.8.0** - - Merge `new-feature-6-18-2026` — code-complete, 342/342 tests passing, - flake8/black clean, and CLI-smoke-tested end-to-end as of 2026-08-08 - - Bump version, tag release, publish to PyPI +1. **Cut v1.8.0** — migration state tracking and `--apply` are both merged + to `main` and fully tested, but `pyproject.toml` is still at 1.7.2 and + nothing has been tagged/published to PyPI yet. -2. **`--apply` flag** - - Closes the "generated vs. applied" gap called out in README's - Migration State Tracking known limitation - - Should auto-call the equivalent of `--record-history` on success - -3. **Native `--fail-on-destructive` CLI flag** +2. **Native `--fail-on-destructive` CLI flag** - Promote the GitHub Action's destructive-detection behavior into `command.py` itself so non-Action CI users (CircleCI, GitLab, plain scripts) get it too +3. **Reconcile `--promote`'s direction with `--apply`** before wiring the + two together — see the Planned/Backlog table above. + 4. **Post v1.8.0:** Enterprise tier planning - Design licensing system - Plan hosted features @@ -342,6 +342,6 @@ without a dedicated version bump — not tied to a CHANGELOG entry. --- -**Document version:** Updated 2026-08-08 (reconciled with actual repo/code state) +**Document version:** Updated 2026-08-08 (reconciled with actual repo/code state, incl. `--apply`) **Last updated by:** Claude (with Leo) **Repository:** https://github.com/postgresql-tools/migra diff --git a/README.md b/README.md index e02b320..f867688 100644 --- a/README.md +++ b/README.md @@ -278,13 +278,13 @@ rollback tracking. ### ⚠ Known Limitation -`migradiff_history` records that a migration was **generated/reviewed** -for a target database, not that the SQL was necessarily *executed* by -MigraDiff itself (MigraDiff does not apply migrations to live databases -today — it only diffs and generates SQL). Recording true "applied" state -requires your deployment pipeline to also call -`migra --record-history` (or the future `--apply` flag, not yet -implemented) after running the generated SQL. +Without `--apply` (below), `migradiff_history` only records that a +migration was **generated/reviewed** for a target database, not that the +SQL was necessarily *executed*. If you generate SQL and pipe it to `psql` +yourself, calling `migra --record-history` only tells MigraDiff "this +migration was proposed" — it has no way to know whether your `psql` step +actually succeeded. Use `--apply` when you want MigraDiff itself to run +the migration and only record history on confirmed success. Be explicit about this in your pipeline so you don't assume false guarantees about whether a migration has actually been applied. @@ -329,6 +329,36 @@ Use `--env-label` to tag the entry: migra --record-history --env-label staging postgres://db_a postgres://db_b ``` +### Applying Migrations (`--apply`) + +`--apply` executes the generated migration directly against `dburl_from` +(the first positional argument — "the database you want to migrate", same +database the README's basic usage example pipes to `psql`) instead of only +printing it: + +```bash +migra --apply postgres://db_production postgres://db_branch +``` + +On success, MigraDiff automatically records the migration in +`dburl_from`'s `migradiff_history` table — you don't need to also pass +`--record-history`. If any statement fails, the whole migration is rolled +back as a single transaction (nothing is partially applied) and **nothing +is recorded**, since it wasn't actually applied. The command exits non-zero +(exit code 4) so pipelines can detect the failure. + +```bash +migra --apply --env-label prod postgres://db_production postgres://db_branch +``` + +`--apply` respects the same safety gates as everything else: destructive +statements are blocked unless `--force-destructive` (or `--unsafe`) is +given, and the block happens *before* anything is executed. + +`--apply` is not supported with `--from-file` (there's no live database to +apply to — the schema files get loaded into temporary throwaway databases) +or with `--promote` (not implemented yet). + ### Multi-Environment Promotion (`--promote`) `--promote` generates migrations along a chain of environments, with diff --git a/migra/command.py b/migra/command.py index 7bee21a..0751d0c 100644 --- a/migra/command.py +++ b/migra/command.py @@ -553,6 +553,16 @@ def parse_args(args): help="Generate a PostgreSQL migration from a plain English description." " Optionally provide the description as an argument.", ) + parser.add_argument( + "--apply", + dest="apply", + action="store_true", + default=False, + help="Execute the generated migration against dburl_from (the database" + " being migrated), instead of only printing it. On success, automatically" + " records the migration in dburl_from's migradiff_history table — no need" + " to also pass --record-history. Not supported with --from-file or --promote.", + ) parser.add_argument( "--record-history", dest="record_history", @@ -641,12 +651,77 @@ def _resolve_env_alias(key): ) +def _apply_migration(dburl_from, statements, sql_output, rollback_sql, env_label): + """Execute `statements` against dburl_from in a single transaction, then + record the migration in dburl_from's migradiff_history table. + + Returns a dict describing the outcome; never raises. Execution and + history-recording are separate try/excepts, since a failure to record + history after a successful apply is a (loud) warning, not an apply + failure — the schema change already happened. + """ + from sqlbag import S as _S + + result = { + "applied": False, + "target": redact_credentials(dburl_from), + "statement_count": len(statements), + "error": None, + "history_recorded": False, + "history_error": None, + } + + try: + with _S(dburl_from) as s: + for stmt in statements: + s.execute(text(stmt)) + except Exception as e: + result["error"] = str(e) + return result + + result["applied"] = True + + from .history import compute_migration_hash, ensure_history_table, record_applied + + try: + with _S(dburl_from) as s: + ensure_history_table(s) + record_applied( + s, + migration_hash=compute_migration_hash(sql_output), + forward_sql=sql_output, + rollback_sql=rollback_sql, + environment_label=env_label, + ) + result["history_recorded"] = True + except Exception as e: + result["history_error"] = str(e) + + return result + + def run(args, out=None, err=None): if not out: out = sys.stdout # pragma: no cover if not err: err = sys.stderr # pragma: no cover + if args.apply and args.from_file: + print( + "MigraDiff: --apply is not supported with --from-file" + " (there is no live database to apply to — dburl_from is a" + " temporary database loaded from the dump file).", + file=err, + ) + return 1 + + if args.apply and args.promote: + print( + "MigraDiff: --apply is not supported with --promote yet.", + file=err, + ) + return 1 + # --status / --history: display migration history if args.status is not None or args.history is not None: from .history import ensure_history_table, get_history @@ -1445,9 +1520,18 @@ def _run_inner(args, out=None, err=None): ) sql_output = None + apply_result = None try: if statements: sql_output = modified_statements.sql + if args.apply: + apply_result = _apply_migration( + args.dburl_from, + modified_statements, + sql_output, + rollback_result["text"] if rollback_result else None, + args.env_label, + ) if args.output == "json": json_out = format_json_output( statements, @@ -1485,6 +1569,12 @@ def _run_inner(args, out=None, err=None): "generated_at": advisory_result.get("generated_at", ""), } json_out = json_mod.dumps(data, indent=2) + if apply_result is not None: + import json as json_mod + + data = json_mod.loads(json_out) + data["apply"] = apply_result + json_out = json_mod.dumps(data, indent=2) print(json_out, file=out) elif args.force_utf8: print(modified_statements.sql.encode("utf8"), file=out) @@ -1501,6 +1591,32 @@ def _run_inner(args, out=None, err=None): print(file=out) print("--- Performance Advisory ---", file=out) print(advisory_result["text"], file=out) + if apply_result is not None and args.output != "json": + print(file=out) + if apply_result["applied"]: + print( + "-- Applied {} statement(s) to {}".format( + apply_result["statement_count"], + apply_result["target"], + ), + file=out, + ) + if not apply_result["history_recorded"]: + print( + "-- WARNING: applied successfully, but failed to" + " record migration history: {}".format( + apply_result["history_error"] + ), + file=err, + ) + else: + print( + "-- ERROR: --apply failed, no changes were committed" + " to {}: {}".format( + apply_result["target"], apply_result["error"] + ), + file=err, + ) elif args.output == "json": json_out = format_json_output( statements, @@ -1562,8 +1678,11 @@ def _run_inner(args, out=None, err=None): args._generated_sql = sql_output if statements else None args._generated_rollback = rollback_result["text"] if rollback_result else None - # --record-history: record the generated migration - if args.record_history and statements and args.dburl_target: + # --record-history: record the generated migration. Skipped when --apply + # already ran, since _apply_migration() already recorded it against + # dburl_from (the database that was actually changed) — recording it + # again here against dburl_target would be a second, misleading entry. + if args.record_history and not args.apply and statements and args.dburl_target: from .history import ( compute_migration_hash, ensure_history_table, @@ -1589,6 +1708,9 @@ def _run_inner(args, out=None, err=None): file=err, ) + if apply_result is not None and not apply_result["applied"]: + return 4 + if not statements: return 0 diff --git a/status.md b/status.md index e9091c3..506f1ff 100644 --- a/status.md +++ b/status.md @@ -3,21 +3,28 @@ Snapshot of current work-in-progress on this branch. Update as work progresses; this is not a changelog (see CHANGELOG.md for that). -## Current branch: `new-feature-6-18-2026` +## Current branch: `feature/apply-flag` (off `main` at `46c2447`) + +Migration state tracking (`--status`/`--history`/`--record-history`/`--promote`/ +`--record-rollback`) and the CI-trigger/SQLAlchemy-2.x fixes are already merged to `main` +(PRs #8, #9, #10) — see git log, not this file, for that history. **In progress / uncommitted:** -- Modified: `CHANGELOG.md`, `README.md`, `migra/command.py` — migration state - tracking (`--status`/`--history`/`--record-history`/`--promote`/`--record-rollback`) -- New: `LICENSING.md` (unrelated), `migra/history.py`, `tests/test_command_promote.py`, - `tests/test_command_rollback_tracking.py`, `tests/test_history.py` -- Also uncommitted (not part of this task): `PROJECT_PLAN.md`, `PROJECT_PLAN2.md` - updated 2026-08-08 to reflect actual shipped state (they were stale, describing - v1.5.1 as current when v1.7.2 was already released) +- Modified: `CHANGELOG.md`, `README.md`, `migra/command.py`, `CLAUDE.md`, + `PROJECT_PLAN.md`, `PROJECT_PLAN2.md` — new `--apply` flag +- New: `tests/test_command_apply.py` + +**What `--apply` does:** executes the generated migration against `dburl_from` in a single +transaction instead of only printing it; on success, automatically records it in `dburl_from`'s +`migradiff_history` table (no need to also pass `--record-history`); on failure, rolls back +everything and records nothing, exits 4. Rejected up front when combined with `--from-file` or +`--promote` — see CLAUDE.md's "Migration state tracking" section for the from/target direction +reasoning. -**Verified 2026-08-08:** all 32 new tests pass against a real Postgres instance; -full suite is 342 passed / 2 skipped; flake8 and black are clean; CLI smoke-tested -end-to-end (`--status`, `--record-history`, `--promote`, `--record-rollback` -round-tripped correctly). Feature is code-complete. +**Verified 2026-08-08:** 11 new tests pass (including a direct atomicity test against +`_apply_migration()` that forces a mid-migration failure); full suite is 353 passed / 2 skipped, +no regressions; flake8 and black clean; also verified against SQLAlchemy 2.0.51 in an isolated +venv (no other raw-SQL issues found). -**Next steps:** merge this branch and release as v1.8.0 (see PROJECT_PLAN.md -"Next Steps" for the follow-on backlog: `--apply`, native `--fail-on-destructive`). +**Next steps:** commit, push, open PR. Known follow-up (not done here): reconcile `--promote`'s +from/to direction with `--apply` before wiring the two together (see PROJECT_PLAN.md backlog). diff --git a/tests/test_command_apply.py b/tests/test_command_apply.py new file mode 100644 index 0000000..fb07802 --- /dev/null +++ b/tests/test_command_apply.py @@ -0,0 +1,246 @@ +from __future__ import unicode_literals + +import io + +from sqlalchemy import text +from sqlbag import S, temporary_database + + +def outs(): + return io.StringIO(), io.StringIO() + + +def _create_table(db_url, table_name, columns="id int primary key"): + with S(db_url) as s: + s.execute(text("CREATE TABLE {} ({});".format(table_name, columns))) + + +def _table_exists(db_url, table_name): + with S(db_url) as s: + row = s.execute( + text( + "SELECT 1 FROM information_schema.tables" + " WHERE table_name = :table_name" + ), + {"table_name": table_name}, + ).fetchone() + return row is not None + + +class TestApplyHappyPath: + def test_apply_creates_table_in_dburl_from(self): + with temporary_database(host="localhost") as from_db, temporary_database( + host="localhost" + ) as target_db: + _create_table(target_db, "widgets") + assert not _table_exists(from_db, "widgets") + + from migra.command import parse_args, run + + args = parse_args(["--apply", from_db, target_db]) + out, err = outs() + status = run(args, out=out, err=err) + + assert status == 2 + assert "-- Applied " in out.getvalue() + assert "statement(s)" in out.getvalue() + assert _table_exists(from_db, "widgets") + + def test_apply_records_history_against_dburl_from(self): + with temporary_database(host="localhost") as from_db, temporary_database( + host="localhost" + ) as target_db: + _create_table(target_db, "widgets") + + from migra.command import parse_args, run + + args = parse_args(["--apply", from_db, target_db]) + out, err = outs() + status = run(args, out=out, err=err) + assert status == 2 + + from migra.history import ensure_history_table, get_history + + with S(from_db) as s: + entries = get_history(s) + assert len(entries) == 1 + assert entries[0]["rollback_status"] == "not_attempted" + assert "CREATE TABLE" in entries[0]["forward_sql"].upper() + + # A plain --record-history (without --apply) should NOT also have + # written a second entry into dburl_target's history table -- in + # fact its history table should never even have been created. + with S(target_db) as s: + ensure_history_table(s) + target_entries = get_history(s) + assert target_entries == [] + + def test_apply_with_env_label(self): + with temporary_database(host="localhost") as from_db, temporary_database( + host="localhost" + ) as target_db: + _create_table(target_db, "widgets") + + from migra.command import parse_args, run + + args = parse_args(["--apply", "--env-label", "prod", from_db, target_db]) + out, err = outs() + status = run(args, out=out, err=err) + assert status == 2 + + from migra.history import get_history + + with S(from_db) as s: + entries = get_history(s) + assert entries[0]["environment_label"] == "prod" + + def test_apply_json_output(self): + with temporary_database(host="localhost") as from_db, temporary_database( + host="localhost" + ) as target_db: + _create_table(target_db, "widgets") + + from migra.command import parse_args, run + import json + + args = parse_args(["--apply", "--output", "json", from_db, target_db]) + out, err = outs() + status = run(args, out=out, err=err) + assert status == 2 + + data = json.loads(out.getvalue()) + assert data["apply"]["applied"] is True + assert data["apply"]["statement_count"] == len(data["statements"]) + assert data["apply"]["history_recorded"] is True + assert _table_exists(from_db, "widgets") + + +class TestApplyNoChanges: + def test_no_diff_is_a_noop(self): + with temporary_database(host="localhost") as from_db, temporary_database( + host="localhost" + ) as target_db: + from migra.command import parse_args, run + + args = parse_args(["--apply", from_db, target_db]) + out, err = outs() + status = run(args, out=out, err=err) + + assert status == 0 + assert "Applied" not in out.getvalue() + # Nothing to apply, so the history table should never even be + # created. + assert not _table_exists(from_db, "migradiff_history") + + +class TestApplySafety: + def test_destructive_change_blocked_without_force(self): + with temporary_database(host="localhost") as from_db, temporary_database( + host="localhost" + ) as target_db: + _create_table(from_db, "widgets") + # target_db has no "widgets" table -> dropping it would be + # destructive. + + from migra.command import parse_args, run + + args = parse_args(["--apply", from_db, target_db]) + out, err = outs() + status = run(args, out=out, err=err) + + assert status == 1 + assert "Destructive operations detected" in err.getvalue() + # The table must still exist -- --apply must never execute + # anything once safe-mode has already rejected the migration. + assert _table_exists(from_db, "widgets") + + def test_destructive_change_applied_with_force_destructive(self): + with temporary_database(host="localhost") as from_db, temporary_database( + host="localhost" + ) as target_db: + _create_table(from_db, "widgets") + + from migra.command import parse_args, run + + args = parse_args(["--apply", "--force-destructive", from_db, target_db]) + out, err = outs() + status = run(args, out=out, err=err) + + assert status == 2 + assert not _table_exists(from_db, "widgets") + + +class TestApplyIncompatibleFlags: + def test_rejected_with_from_file(self): + from migra.command import parse_args, run + + args = parse_args(["--apply", "--from-file", "a.sql", "b.sql"]) + out, err = outs() + status = run(args, out=out, err=err) + assert status == 1 + assert "--apply is not supported with --from-file" in err.getvalue() + + def test_rejected_with_promote(self): + from migra.command import parse_args, run + + args = parse_args(["--apply", "--promote", "dev:staging"]) + out, err = outs() + status = run(args, out=out, err=err) + assert status == 1 + assert "--apply is not supported with --promote" in err.getvalue() + + +class TestApplyMigrationHelper: + """Unit tests against _apply_migration() directly, so we can deterministically + force a mid-migration failure and assert atomicity -- something that's hard + to trigger reliably by going through the full diff-generation pipeline.""" + + def test_failed_statement_rolls_back_and_records_nothing(self): + with temporary_database(host="localhost") as db_url: + from migra.command import _apply_migration + + result = _apply_migration( + db_url, + [ + "CREATE TABLE ok_table (id int primary key);", + "THIS IS NOT VALID SQL;", + ], + sql_output="CREATE TABLE ok_table (id int primary key);\n\n" + "THIS IS NOT VALID SQL;\n\n", + rollback_sql=None, + env_label=None, + ) + + assert result["applied"] is False + assert result["error"] is not None + assert result["history_recorded"] is False + + # The whole transaction must have rolled back -- ok_table must + # NOT exist despite being a valid statement that ran first. + assert not _table_exists(db_url, "ok_table") + assert not _table_exists(db_url, "migradiff_history") + + def test_successful_apply_records_history(self): + with temporary_database(host="localhost") as db_url: + from migra.command import _apply_migration + + sql = "CREATE TABLE widgets (id int primary key);" + result = _apply_migration( + db_url, + [sql], + sql_output=sql, + rollback_sql="DROP TABLE widgets;", + env_label="staging", + ) + + assert result["applied"] is True + assert result["history_recorded"] is True + assert _table_exists(db_url, "widgets") + + from migra.history import get_history + + with S(db_url) as s: + entries = get_history(s) + assert len(entries) == 1 + assert entries[0]["rollback_sql"] == "DROP TABLE widgets;" + assert entries[0]["environment_label"] == "staging"