feat: make the SQL engine pluggable and reconstruct types from schema

AGENTS.md

@@ -56,4 +56,3 @@ This package converts Pro Cycling Manager CDB binary database files to and from
 ## References
 
 - API usage and examples: [README.md](README.md)
-- Build/export behavior note: [build-notes.md](memories/repo/build-notes.md)

README.md

@@ -36,8 +36,8 @@ The conversion is **lossless**: a full `cdb → sqlite → cdb` round-trip prese
 - **CLI included** — convert files without writing any code; direction is auto-detected.
 - **Lossless round-trip** — table flags, column order, and data types survive an export/reopen cycle.
 - **Optional relational schema** — reconstruct `PRIMARY KEY` / `FOREIGN KEY` constraints for JOINs and ER diagrams, without breaking the round-trip.
-- **Isomorphic** — runs in Node.js and in the browser via [sql.js](https://github.com/sql-js/sql.js).
-- **Lightweight** — the library's own code is ~28 kB, with only `pako` and `sql.js` as dependencies.
+- **Node-first, browser-capable** — the CLI and default `better-sqlite3` engine target Node.js; an optional `sql.js` (WASM) engine covers the browser.
+- **Lightweight** — the library's own code is ~28 kB. `better-sqlite3` is a hard dependency; the `sql.js` (WASM) engine is optional and installed separately.
 - **TypeScript-first** — native type definitions and full IDE support.
 - **Tree-shakeable** — pure functions, no side effects, ESM + CommonJS builds.
 
@@ -48,7 +48,7 @@ npm install cdb-converter
 ```
 
 > [!NOTE]
-> Requires **Node.js 22 or newer**. In the browser, `sql.js` loads its WebAssembly runtime on demand.
+> Requires **Node.js 22 or newer**. The CLI and the default `better-sqlite3` engine are ready to use out of the box — `better-sqlite3` is installed automatically as a dependency. For the browser, install `sql.js` instead — it loads its WebAssembly runtime on demand.
 
 The fastest way to try it is the CLI:
 
@@ -83,25 +83,23 @@ npx cdb-converter --version
 | `.cdb`            | CDB → SQLite | `<input>.sqlite` |
 | `.sqlite` / `.db` | SQLite → CDB | `<input>.cdb`    |
 
-| Option              | Effect                                                                                             |
-| ------------------- | -------------------------------------------------------------------------------------------------- |
+| Option              | Effect                                                                                                                      |
+| ------------------- | --------------------------------------------------------------------------------------------------------------------------- |
 | `-n`, `--normalize` | (CDB → SQLite only) reconstruct PK/FK constraints from PCM naming conventions. See [Normalized schema](#normalized-schema). |
-| `--index-fk`        | Implies `--normalize`; also indexes every FK column for faster JOINs (roughly doubles output size). |
+| `--index-fk`        | Implies `--normalize`; also indexes every FK column for faster JOINs (roughly doubles output size).                         |
 
 ## Library usage
 
 ### CDB to SQLite
 
 ```typescript
 import fs from "node:fs";
-import initSqlJs from "sql.js";
+import { betterSqlite3Engine } from "cdb-converter/engines/better-sqlite3";
 import { cdbToSql } from "cdb-converter";
 
-const SQL = await initSqlJs();
-
 // Read and convert a CDB file
 const cdbBuffer = fs.readFileSync("save.cdb");
-const db = cdbToSql(cdbBuffer, SQL);
+const db = cdbToSql(cdbBuffer, betterSqlite3Engine);
 
 // Query it like any SQLite database
 const result = db.exec("SELECT * FROM Teams LIMIT 5");
@@ -112,14 +110,14 @@ fs.writeFileSync("save.sqlite", db.export());
 ```
 
 > [!IMPORTANT]
-> You must pass the initialized `sql.js` module returned by `initSqlJs()`. This library does not initialize `sql.js` for you: that setup is asynchronous and environment-specific (the caller decides how the wasm file is loaded in Node.js or the browser).
+> You must pass a `SqlEngine`. `cdb-converter/engines/better-sqlite3` (requires the `better-sqlite3` package) is the default for Node.js; `cdb-converter/engines/sql-js` (requires the `sql.js` package) works in the browser but needs an `await` to initialize its WASM runtime. See [Using a different SQLite engine](#using-a-different-sqlite-engine).
 
 ### Normalized schema
 
 By default the SQLite output is a flat mirror of the CDB tables, with no relational constraints. Pass `{ normalize: true }` to reconstruct `PRIMARY KEY` and `FOREIGN KEY` constraints from the PCM naming conventions (`ID{table}` identity columns and `fkID{target}` references), turning the export into a proper relational database — ready for JOINs, entity-relationship diagrams, and schema introspection tools.
 
 ```typescript
-const db = cdbToSql(cdbBuffer, SQL, { normalize: true });
+const db = cdbToSql(cdbBuffer, betterSqlite3Engine, { normalize: true });
 
 // Relationships are now navigable:
 db.exec(`
@@ -138,24 +136,25 @@ Notes:
 
 ```typescript
 // Lean: constraints only (~+40% size)
-cdbToSql(cdbBuffer, SQL, { normalize: true });
+cdbToSql(cdbBuffer, betterSqlite3Engine, { normalize: true });
 
 // Heavier, faster JOINs: also index FK columns (~2x size)
-cdbToSql(cdbBuffer, SQL, { normalize: true, indexForeignKeys: true });
+cdbToSql(cdbBuffer, betterSqlite3Engine, {
+  normalize: true,
+  indexForeignKeys: true,
+});
 ```
 
 ### SQLite to CDB
 
 ```typescript
 import fs from "node:fs";
-import initSqlJs from "sql.js";
+import { betterSqlite3Engine } from "cdb-converter/engines/better-sqlite3";
 import { sqlToCdb } from "cdb-converter";
 
-const SQL = await initSqlJs();
-
 // Load a SQLite database and convert back to CDB
 const sqliteBuffer = fs.readFileSync("save.sqlite");
-const db = new SQL.Database(sqliteBuffer);
+const db = new betterSqlite3Engine.Database(sqliteBuffer);
 
 const cdbBuffer = sqlToCdb(db); // automatically compressed
 fs.writeFileSync("save.cdb", Buffer.from(cdbBuffer));
@@ -178,8 +177,9 @@ const decompressed = decompressCdb(compressed); // accepts compressed or raw inp
 <script src="https://cdn.jsdelivr.net/npm/sql.js@1.14.1/dist/sql-wasm.js"></script>
 <script type="module">
   import { cdbToSql } from "https://cdn.jsdelivr.net/npm/cdb-converter/+esm";
+  import { createSqlJsEngine } from "https://cdn.jsdelivr.net/npm/cdb-converter/+esm/engines/sql-js";
 
-  const SQL = await initSqlJs({
+  const SQL = await createSqlJsEngine({
     locateFile: (file) =>
       `https://cdn.jsdelivr.net/npm/sql.js@1.14.1/dist/${file}`,
   });
@@ -195,23 +195,34 @@ const decompressed = decompressCdb(compressed); // accepts compressed or raw inp
 
 ## API reference
 
-### `cdbToSql(cdbBuffer, SQL, options?): Database`
+### `cdbToSql(cdbBuffer, SQL, options?): SqlDatabase`
 
 Convert CDB binary data into a SQLite database instance.
 
 - **`cdbBuffer`** — `ArrayBuffer | Uint8Array`, raw CDB data (compressed or uncompressed).
-- **`SQL`** — `SqlJsStatic`, the module returned by `initSqlJs()`.
+- **`SQL`** — a `SqlEngine`. Use `betterSqlite3Engine` from `cdb-converter/engines/better-sqlite3` (Node.js) or `createSqlJsEngine()` from `cdb-converter/engines/sql-js` (browser). See [Using a different SQLite engine](#using-a-different-sqlite-engine).
 - **`options.normalize`** — `boolean` (default `false`). Reconstruct PK/FK constraints from PCM naming conventions. See [Normalized schema](#normalized-schema).
 - **`options.indexForeignKeys`** — `boolean` (default `false`). When normalizing, also index every FK column for faster JOINs (roughly doubles the output size).
-- **returns** — a `sql.js` `Database` with the CDB tables loaded.
+- **returns** — a `SqlDatabase` with the CDB tables loaded.
 
 ### `sqlToCdb(db): ArrayBuffer`
 
 Convert a SQLite database back to CDB binary format (automatically compressed).
 
-- **`db`** — a `sql.js` `Database` instance.
+- **`db`** — a `SqlDatabase` instance.
 - **returns** — compressed CDB binary data as an `ArrayBuffer`.
 
+### Using a different SQLite engine
+
+The library's public API isn't tied to any single SQLite implementation — it's typed against the minimal, self-contained `SqlEngine`/`SqlDatabase` interfaces exported from the package root, so any object matching that shape works. Two engines ship with the package as optional subpaths (their underlying SQLite package must be installed separately, so consumers who don't need one don't pay for it):
+
+- **`cdb-converter/engines/better-sqlite3`** — exports `betterSqlite3Engine`. The default for Node.js and what the CLI uses; requires `npm install better-sqlite3` (a native dependency).
+- **`cdb-converter/engines/sql-js`** — exports `createSqlJsEngine()` (async). The only option that runs in the browser; requires `npm install sql.js`.
+
+Other engines can be wired up with a hand-written adapter matching `SqlEngine`/`SqlDatabase`:
+
+- [Node.js — swapping the SQLite engine](./samples/node-sqlite-engine/) — a sample adapter for Node's built-in [`node:sqlite`](https://nodejs.org/api/sqlite.html), not shipped yet since its API is still experimental.
+
 ### `compressCdb(data): ArrayBuffer`
 
 Compress CDB data using zlib deflate. Accepts `ArrayBuffer | Uint8Array`.
@@ -269,20 +280,20 @@ A full `cdb → sqlite → cdb` round-trip on a real ~60k-row database stays wel
 
 Normalization is opt-in and costs only what you ask for (measured against the default conversion, ~60k rows):
 
-| Mode                                          | Conversion time | Output size |
-| --------------------------------------------- | --------------- | ----------- |
-| Default (flat)                                | baseline        | baseline    |
-| `normalize`                                   | +~10%           | +~40%       |
-| `normalize` + `indexForeignKeys`              | +~40%           | +~130%      |
+| Mode                             | Conversion time | Output size |
+| -------------------------------- | --------------- | ----------- |
+| Default (flat)                   | baseline        | baseline    |
+| `normalize`                      | +~10%           | +~40%       |
+| `normalize` + `indexForeignKeys` | +~40%           | +~130%      |
 
 See **[bench/README.md](bench/README.md)** for the full per-fixture numbers, the bundle breakdown, and how to reproduce them (`npm run bench`).
 
 ## Samples
 
 Runnable examples live in the [samples](./samples/) folder:
 
-- [Browser](./samples/browser/) — convert a `.cdb` file to SQLite directly in the browser.
-- [Node.js — CDB to SQLite](./samples/node-cdb-to-sql/) — convert a `.cdb` file into a `.sqlite` file.
+- [Browser](./samples/browser/) — convert a `.cdb` file to SQLite directly in the browser, using the `sql.js` engine.
+- [Node.js — CDB to SQLite](./samples/node-cdb-to-sql/) — convert a `.cdb` file into a `.sqlite` file, using the default `better-sqlite3` engine.
 - [Node.js — SQLite to CDB](./samples/node-sql-to-cdb/) — convert a `.sqlite` or `.db` file back into a `.cdb` file.
 
 ## License

bench/README.md

@@ -13,8 +13,10 @@ npm run bench                          # all official fixtures
 npm run bench -- --outputJson out.json # machine-readable results
 ```
 
-Each case is warmed up before timing, so a cold outlier (JIT warm-up, sql.js/WASM init)
-does not skew the result.
+Each case is warmed up before timing, so a cold outlier (JIT warm-up, native binding load)
+does not skew the result. Benchmarks run against the `better-sqlite3` engine, matching what
+the CLI uses in Node.js — see [Bundle size](#bundle-size) for how the `sql.js` (WASM) engine
+compares.
 
 ## Conversion performance
 
@@ -23,18 +25,21 @@ Round-trip timings against the official Pro Cycling Manager databases (`mean` re
 
 | Fixture                  | Input (CDB) | Tables |   Rows | cdbToSql | sqlToCdb | Round-trip |
 | ------------------------ | ----------: | -----: | -----: | -------: | -------: | ---------: |
-| OfficialRelease-2014.cdb |      355 kB |    136 | 31,741 | 131.7 ms | 119.6 ms |   251.3 ms |
-| OfficialRelease-2018.cdb |      566 kB |    150 | 62,127 | 167.8 ms | 198.5 ms |   366.3 ms |
-| OfficialRelease-2019.cdb |      610 kB |    158 | 64,560 | 177.2 ms | 210.0 ms |   387.2 ms |
-| OfficialRelease-2021.cdb |      417 kB |    147 | 36,672 | 133.1 ms | 128.1 ms |   261.2 ms |
-| OfficialRelease-2025.cdb |      441 kB |    149 | 34,691 | 145.3 ms | 144.8 ms |   290.1 ms |
+| OfficialRelease-2014.cdb |      355 kB |    136 | 31,741 | 107.2 ms | 100.5 ms |   207.7 ms |
+| OfficialRelease-2018.cdb |      566 kB |    150 | 62,127 | 141.5 ms | 171.1 ms |   312.6 ms |
+| OfficialRelease-2019.cdb |      610 kB |    158 | 64,560 | 144.9 ms | 179.5 ms |   324.3 ms |
+| OfficialRelease-2021.cdb |      417 kB |    147 | 36,672 | 109.3 ms | 109.3 ms |   218.6 ms |
+| OfficialRelease-2025.cdb |      441 kB |    149 | 34,691 | 122.6 ms | 125.8 ms |   248.4 ms |
 
 - **cdbToSql** — decompress + parse the CDB binary into an in-memory SQLite database.
 - **sqlToCdb** — serialize the SQLite database back to compressed CDB bytes.
-- **Round-trip** — `cdbToSql + sqlToCdb`. sql.js `db.export()` (SQLite → bytes) is
-  negligible (~0.5 ms) and omitted from the total.
+- **Round-trip** — `cdbToSql + sqlToCdb`. `db.export()` (SQLite → bytes) is
+  negligible (~0.3 ms) and omitted from the total.
 
-Most of the time is spent in binary parsing/writing, not in SQLite itself.
+Most of the time is spent in binary parsing/writing, not in SQLite itself. Native
+`better-sqlite3` runs this ~15-17% faster end-to-end than the previous `sql.js` (WASM)
+baseline, since it avoids per-call WASM boundary overhead on the large number of small
+`run`/`exec` calls the conversion makes.
 
 ## Normalization overhead
 
@@ -47,11 +52,11 @@ their cost against the default flat conversion.
 
 | Fixture                  |  Default | `normalize` |  `+ indexForeignKeys` |
 | ------------------------ | -------: | ----------: | --------------------: |
-| OfficialRelease-2014.cdb | 132.7 ms | 148.8 ms (+12%) |    187.4 ms (+41%) |
-| OfficialRelease-2018.cdb | 172.5 ms | 194.9 ms (+13%) |    234.4 ms (+36%) |
-| OfficialRelease-2019.cdb | 173.3 ms | 188.0 ms (+8%)  |    240.8 ms (+39%) |
-| OfficialRelease-2021.cdb | 133.1 ms | 145.9 ms (+10%) |    194.6 ms (+46%) |
-| OfficialRelease-2025.cdb | 137.8 ms | 149.5 ms (+8%)  |    192.1 ms (+39%) |
+| OfficialRelease-2014.cdb | 107.2 ms | 122.8 ms (+14%) |    140.6 ms (+31%) |
+| OfficialRelease-2018.cdb | 141.5 ms | 151.0 ms (+7%)  |    190.1 ms (+34%) |
+| OfficialRelease-2019.cdb | 144.9 ms | 160.6 ms (+11%) |    189.6 ms (+31%) |
+| OfficialRelease-2021.cdb | 109.3 ms | 114.7 ms (+5%)  |    149.2 ms (+37%) |
+| OfficialRelease-2025.cdb | 122.6 ms | 124.1 ms (+1%)  |    150.1 ms (+23%) |
 
 ### Output size (`db.export()`)
 
@@ -73,16 +78,18 @@ their cost against the default flat conversion.
 
 ## Bundle size
 
-The converter's own code is tiny; the footprint you actually ship is dominated by the
-SQLite engine it depends on.
-
-| Component                       | Size                                     |
-| ------------------------------- | ---------------------------------------- |
-| cdb-converter (published, gzip) | **~28 kB** (npm tarball)                 |
-| `sql.js` WebAssembly runtime    | ~644 kB (`sql-wasm.wasm`, loaded lazily) |
-| `pako` (zlib deflate/inflate)   | small, tree-shakeable                    |
-
-The library itself adds only a few kilobytes. The SQLite WASM binary is the real weight,
-and you would pay for it with any SQLite-in-JS approach. In the browser the `.wasm` is
-fetched on demand (not part of your JS bundle); in Node.js it is loaded from
-`node_modules` at runtime.
+The converter's own code is tiny; the footprint you actually ship depends on which SQLite
+engine you use.
+
+| Component                                        | Size                                                               |
+| ------------------------------------------------- | ------------------------------------------------------------------- |
+| cdb-converter (published, gzip)                  | **~28 kB** (npm tarball)                                           |
+| `better-sqlite3` (default, Node)                 | native addon, compiled at install time — no JS/WASM bundle weight |
+| `sql.js` WebAssembly runtime (browser, optional) | ~644 kB (`sql-wasm.wasm`, loaded lazily)                           |
+| `pako` (zlib deflate/inflate)                    | small, tree-shakeable                                              |
+
+The library itself adds only a few kilobytes. `better-sqlite3` — the default engine used
+by the CLI and Node consumers — is a native addon: it costs an install-time compile step
+(or a prebuilt binary), not JS/WASM bundle size. `sql.js` remains available as an optional
+engine (`cdb-converter/engines/sql-js`) for the browser, where its `.wasm` is fetched on
+demand rather than bundled into your JS.

bench/roundtrip.bench.ts

@@ -4,14 +4,15 @@
 //   - cdbToSql                     : decompress + parse CDB -> in-memory SQLite
 //   - cdbToSql (normalize)         : same, plus reconstructed PK/FK constraints
 //   - cdbToSql (normalize+fkIndex) : same, plus an index on every FK column
-//   - export                       : sql.js db.export() (SQLite -> bytes)
+//   - export                       : better-sqlite3 db.export() (SQLite -> bytes)
 //   - sqlToCdb                     : SQLite -> compressed CDB bytes
 //
+// Uses the better-sqlite3 engine, matching what the CLI runs in Node.
 
 import { readFileSync } from "node:fs";
 import { basename } from "node:path";
-import initSqlJs from "sql.js";
 import { bench, describe } from "vitest";
+import { betterSqlite3Engine } from "../src/engines/better-sqlite3";
 import { cdbToSql, sqlToCdb } from "../src/index";
 
 const FIXTURES = [
@@ -22,16 +23,16 @@ const FIXTURES = [
 	"test/fixtures/OfficialRelease-2025.cdb",
 ];
 
-const SQL = await initSqlJs();
+const SQL = betterSqlite3Engine;
 
 for (const path of FIXTURES) {
 	const cdbBytes = readFileSync(path);
 
 	describe(basename(path), () => {
 		// cdbToSql builds a fresh database each run. vitest's setup/teardown are
 		// per-cycle (not per-iteration) hooks, so we close the database inside the
-		// timed function to keep the sql.js/WASM heap from growing across
-		// iterations. close() is negligible next to the parse it measures.
+		// timed function to release native handles between iterations. close() is
+		// negligible next to the parse it measures.
 		bench("cdbToSql", () => {
 			cdbToSql(cdbBytes, SQL).close();
 		});