feat: DB driver breadth — MySQL + SQLite built-ins, hardened protocol#53
Merged
Conversation
…protocol) Locked design decisions from brainstorming 2026-07-21: - Ship MySQL (PyMySQL) + SQLite (stdlib) built-ins - Per-engine [mysql] extra; SQLite ships in core - DTOs + BaseDBDriver mixin (medium hardening) - Both new drivers implement describe_schema (Level-1 + Level-2) Spec: docs/superpowers/specs/active/2026-07-21-db-driver-breadth-design.md
…bClient serializes at boundary
Wires the three DB drivers into the packaging layer: - pyproject.toml: add `mysql` optional-dependency extra (["PyMySQL>=1.1", "jq>=1.6"]); no `sqlite` extra (stdlib). - pyproject.toml: register `mysql` and `sqlite` entry points under `agctl.db_drivers` alongside the existing `postgresql`. - db_client.py: import MySQLDriver + SQLiteDriver, extend BUILTIN_DRIVERS so load_drivers() returns all three. The lazy-import invariant is preserved: psycopg and pymysql are deferred-imported inside each driver's connect()/execute() methods, so populating BUILTIN_DRIVERS at module load does NOT trigger those imports. `import agctl.clients.db_client` succeeds with both heavy deps absent (verified by sys.modules None-sentinel test). Updates one pre-existing test_db_client.py case that used "mysql" as a placeholder for an unknown driver type — mysql is now a real registered driver, so the test would never raise.
…drivers
Updates the user-facing contract and as-built implementation docs to
reflect Task 8's packaging changes:
- DESIGN.md: connection types now list postgresql/mysql/sqlite; module
tree includes db_drivers/{mysql,postgresql,sqlite}.py; pyproject
snippet shows the new mysql extra and the three registered entry
points.
- ARCHITECTURE.md: client tree lists all three db_drivers/*.py files;
DbClient section header + BUILTIN_DRIVERS dict literal updated;
entry-point groups description mentions all three built-ins.
PyMySQL's FieldDescriptorPacket.description() returns 7-tuples per PEP 249
(name, type_code, display_size, internal_size, precision, scale, null_ok),
NOT objects with a .name attribute. The driver's previous desc.name indexing
worked against the FakeCursor test seam (PostgreSQL convention using
types.SimpleNamespace) but raised AttributeError against real PyMySQL --
meaning every db query / db schema call against live MySQL was broken at
runtime.
Fix: index by [0] in all 7 sites in mysql.py (execute, describe_schema x3,
_describe_one_relation x3), matching the SQLite driver's convention. The
PostgreSQL driver is unaffected -- psycopg returns Column objects that DO
have a .name attribute.
Hardened the unit-test seam: _col() in test_mysql_driver.py now returns the
realistic 7-tuple shape, so any future regression of desc.name is caught at
unit-test time rather than waiting for an integration test against live
PyMySQL. All 28 existing MySQL unit tests pass with the more realistic shape.
Bug verification (deferred from Task 6 review):
- pymysql/protocol.py:257-267 confirms description() returns a 7-tuple.
- Standalone repro: indexing a real PEP 249 7-tuple via .name raises
'tuple' object has no attribute 'name'; via [0] returns the column name.
…n-memory)
Exercises the full db query / db assert / db execute / db schema command
surface through the CLI command layer (CliRunner) against real driver
libraries -- catching dialect-specific surprises the FakeCursor unit tests
cannot.
SQLite (3 tests, run unconditionally -- stdlib sqlite3):
- full_cycle: execute->query->assert->schema round-trip on a tempfile DB
(cross-invocation state survives because the file persists; :memory:
would lose data between CliRunner invocations).
- assert_expect_value: --path .status --equals CONFIRMED on a seeded row.
- execute_write_gates: --write required + writable=false rejected (both
unchanged behavior; verifies the gates still hold for the new driver).
MySQL (4 tests, via testcontainers mysql:8; SKIP cleanly if Docker
unavailable or AGCTL_TEST_MYSQL_DSN unreachable):
- full_cycle: parent/child tables with ENUM('new','old') + FK; verifies
rows_affected, FK introspection, enum_values parsing, system-schema
filtering.
- execute_write_visible: INSERT returns rows_affected=1, returning=[],
row appears in subsequent query (committed write visible).
- execute_no_returning_clause: MySQL rejects RETURNING syntax -> surfaces
as ConnectionFailure (exit 2); documents the dialect limitation.
- auto_increment_generated_mapping: AUTO_INCREMENT column maps to
generated="by_default_identity" with default=None (redaction rule).
The MySQL tests would have caught the desc.name bug fixed in the previous
commit (any db query / db schema call against real PyMySQL 7-tuple
description raised AttributeError); they now pass once Docker is available.
Fixture design: require_mysql tries AGCTL_TEST_MYSQL_DSN first (manual/CI),
then spins up a session-cached mysql:8 container. Both paths self-skip on
any failure (Docker down, registry unreachable, pymysql missing) -- never
fails when the service is absent, matching the existing require_postgres
convention.
Added mysql + sqlite connection examples to DESIGN.md §2.1 alongside existing PostgreSQL examples; documented DatabaseConnection.options field. Added §9.1 paragraphs documenting DTOs (WriteResult, SchemaItem, SchemaMatch, ColumnInfo, ForeignKey, UniqueConstraint), the BaseDBDriver mixin, and the optional-capability duck-typing contract. Fix: corrected mysql-replica example that put charset as a top-level field (would fail Pydantic validation); moved under options: where driver-specific extras belong. ARCHITECTURE.md was already correctly synced by Task 8 (commit 369b47f): §3 module map, §8 DB client layer, §10 extension points, §14 deltas.
…Key type, MySQL autocommit crash, docs accuracy)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes the PostgreSQL-only gap in
agctl db *. Ships built-in MySQL (PyMySQL) + SQLite (stdlib) drivers, hardens the contributor extension surface via typed DTOs +BaseDBDrivermixin.docs/superpowers/specs/active/2026-07-21-db-driver-breadth-design.mddocs/superpowers/plans/active/2026-07-21-db-driver-breadth.mdWhat's new
Hardened protocol (
agctl/clients/db_driver_protocol.py):WriteResult,SchemaItem,SchemaMatch,ColumnInfo,ForeignKey,UniqueConstraintBaseDBDrivermixin with_redact_config(scheme-agnostic URL userinfo redaction) and_lazy_import_or_raisehelpersNew drivers:
agctl/clients/db_drivers/mysql.py— PyMySQL-backed;autocommit=False;convert_sql_paramsrewrite; fulldescribe_schemaviainformation_schema(enum literal parser, FK positional pairing, auto_increment → generated mapping)agctl/clients/db_drivers/sqlite.py— stdlibsqlite3; native:nameparams (no rewrite);describe_schemaviasqlite_master+ PRAGMAs with strict identifier validationConfig & packaging:
DatabaseConnection.options: dictfield for driver-specific extras (charset, collation, etc.)[mysql]extra (pip install 'agctl[mysql]'); SQLite ships in core (stdlib)[project.entry-points."agctl.db_drivers"]Refactor:
PostgreSQLDriverinheritsBaseDBDriver; returns DTOs internally;DbClientserializes viadataclasses.asdictat the boundarySchemaMatchfield-order fix in commitcd0369a)Test results
Process
10-task plan executed via subagent-driven development. Each task: implementer subagent → task review (spec + quality) → fix dispatch if needed → re-review. Two issues caught by the review loop:
commentat position 4; original PostgreSQL dict had it last. Becauseagctl/output.pydoesn'tsort_keys, this would have broken byte-equality ofagctl db schemaJSON output. Fixed via field reorder + docstring documenting the invariant.desc.name→desc[0](deferred from Tasks 6/7, fixed in Task 9) — the FakeCursor test seam used objects with.name(PostgreSQL convention), but real PyMySQL returns PEP-249 7-tuples. Integration test surfaced it; fixed at all 7 sites + hardened the unit-test seam to use 7-tuples preventing regression.Docs
optionsfield documentedDeferred Minors (recommended for follow-up polish PR)
10 cosmetic/edge-case items recorded in the implementation ledger; most defensible: MySQL
urllib.parse.unquote(parsed.password)for URL-encoded passwords.Checklist