Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 45 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,49 @@ While it's pre-1.0, minor versions may carry breaking changes.

## [Unreleased]

## [2.0.0] - 2026-08-26

The one intentional breaking bump. The config object changes shape and the factory gets a new
name — both batched here, while the install base is small, so consumers migrate once.

### Changed

- **Config declares both engines and selects the active one** (breaking): `DALConfig.db` is no
longer a `SqliteConfig | PostgresConfig` union with a `mode` discriminant on each block. It is
now a plain selector — `db: 'local' | 'cloud'` — with the two engine configs declared alongside
it under `local` and `cloud`:

```ts
// before (1.x)
await db.connect({ db: { mode: 'local', dataDir: './data' } });
// after (2.0)
await db.connect({ db: 'local', local: { dataDir: './data' } });
```

Every declared block is validated at `connect()`, so a bad cloud `connectionString` is caught at
boot while you're still on local — not the first time you flip to cloud in production. The `mode`
field is gone from `SqliteConfig`/`PostgresConfig` (the selector replaces it). A `db: 'cloud'`
with no `cloud` block throws `ConfigurationError`.

### Added

- **`sqlSwitch()`** is the factory's new name (matches the package, fits the `express()`/`fastify()`
package-as-factory convention). `createDAL` stays as a `@deprecated` alias — one line, no second
code path — and will be removed in 3.0. The default export is `sqlSwitch`, so
`import sqlSwitch from 'sql-switch'` needs no change.
- **`db.reconnect(target?)`**: one primitive for restart / recover-a-wedged-connection / repoint.
No argument re-opens the current engine; `'local'`/`'cloud'` switches to the other declared engine
**without moving data** (that's still `swapEngine()`). Flushes pending writes first, and is
fail-safe — the target engine is built and validated before the current one is torn down, so a bad
target leaves you on the engine you had.

### Migration

- `db: { mode: 'local', ...rest }` → `db: 'local', local: { ...rest }`.
- `db: { mode: 'cloud', connectionString, pool }` → `db: 'cloud', cloud: { connectionString, pool }`.
- Optionally declare both blocks and drive `db` from an env var (`db: process.env.DB_MODE`).
- `createDAL()` keeps working; rename to `sqlSwitch()` at your leisure before 3.0.

## [1.0.1] - 2026-08-21

Tier 1 correctness fixes (packaging + Postgres value integrity + write durability).
Expand Down Expand Up @@ -104,7 +147,8 @@ Initial pre-release of the universal SQLite/PostgreSQL DAL.
crashing, then recovers.
- Bidirectional engine swap => migrate data SQLite files <=> PostgreSQL schemas.

[Unreleased]: https://github.com/creative-softworks/sql-switch/compare/v1.0.1...HEAD
[Unreleased]: https://github.com/creative-softworks/sql-switch/compare/v2.0.0...HEAD
[2.0.0]: https://github.com/creative-softworks/sql-switch/compare/v1.0.1...v2.0.0
[1.0.1]: https://github.com/creative-softworks/sql-switch/compare/v1.0.0...v1.0.1
[1.0.0]: https://github.com/creative-softworks/sql-switch/compare/v0.2.0...v1.0.0
[0.2.0]: https://github.com/creative-softworks/sql-switch/compare/v0.1.0...v0.2.0
Expand Down
140 changes: 88 additions & 52 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,69 +1,103 @@
# sql-switch
<p align="center">
<img src="./assets/readme/hero.svg" width="100%" alt="sql-switch — one fluent API for your data: SQLite in dev, PostgreSQL in prod, with a one-command engine swap between the two">
</p>

Universal hot-swappable database abstraction layer for Node.js. Run SQLite locally, PostgreSQL in production — the same fluent API covers both. Migrate your data between engines with one CLI command, or one function call.
<p align="center">
<a href="https://www.npmjs.com/package/sql-switch"><img src="https://img.shields.io/npm/v/sql-switch?logo=npm&amp;color=3fb950&amp;label=npm" alt="npm version"></a>
<a href="https://www.npmjs.com/package/sql-switch"><img src="https://img.shields.io/npm/types/sql-switch?color=7c8cff" alt="TypeScript types included"></a>
<img src="https://img.shields.io/node/v/sql-switch?color=56d4dd&amp;label=node" alt="supported Node.js versions">
<a href="./LICENSE"><img src="https://img.shields.io/npm/l/sql-switch?color=8b949e&amp;label=license" alt="MIT license"></a>
</p>

## Install
<p align="center">
<b>One fluent API for your data.</b> Run SQLite while you build, PostgreSQL in production —
<br>the same call works on both — then migrate between engines with a single command.
</p>

```bash
npm install sql-switch
```

Also published under the Creative-Softworks scope if you prefer the branded name — it re-exports
this package unchanged, so pick whichever reads better to you:
---

```bash
npm install @creative-softworks/sql-switch
```

The database drivers are **optional peer dependencies** — install only the one for the engine you
run, so a SQLite-only app never pulls in `pg` and its build, and vice versa:
## Quick start

```bash
npm install better-sqlite3 # local mode
npm install pg # cloud mode
npm install sql-switch better-sqlite3
```

Each driver is loaded lazily the first time you `connect()` in that mode, so importing the package
never requires both engines to be present.

## Quick start

```ts
import { createDAL } from 'sql-switch';
import { sqlSwitch } from 'sql-switch';

const db = createDAL();
const db = sqlSwitch();

await db.connect({
db: { mode: 'local', dataDir: './data/databases', wal: true },
db: 'local',
local: { dataDir: './data/databases', wal: true },
collector: { enabled: true, time: 3000 },
});

// read
const settings = await db.schema('antinuke').table('settings').key('guild_123').get();

// write (queued, flushed every 3s in bulk)
// write — queued in RAM, flushed in bulk every 3s
await db.schema('antinuke').table('settings').key('guild_123').set({ strict: true });

// write immediately, bypassing the collector
// read — a queued write is visible to the next get() on the same key
const settings = await db.schema('antinuke').table('settings').key('guild_123').get();

// need it on disk right now? bypass the collector
await db.schema('antinuke').table('settings').key('guild_123').set({ strict: true }).force();
```

## Switch to PostgreSQL
That is the whole surface: `schema → table → key → operation`. The same chain drives every engine.

## Same code, production engine

Declare **both** engines up front and pick the active one with `db` — an env var is the usual
selector. Only the config changes; not one line of your data code does:

```ts
await db.connect({
db: {
mode: 'cloud',
db: process.env.DB_MODE ?? 'local', // 'local' | 'cloud'
local: { dataDir: './data/databases', wal: true },
cloud: {
connectionString: process.env.DATABASE_URL,
pool: { max: 5, statementTimeout: 30_000 },
},
collector: { enabled: true, time: 3000 },
});
```

`statementTimeout` (default 30s, `0` disables) is the ceiling on a single operation. Without one a
query that never answers holds a pool connection for the life of the process — `max` of those and
every later read blocks with no error at all.
Both declared blocks are validated at `connect()`, so a bad cloud `connectionString` is caught at
boot even while you're still running local. Flip engines at runtime — no data move, just repoint the
client — with `db.reconnect('cloud')`; use `db.swapEngine()` when the rows need to travel too.

In local mode each schema is its own `.db` file (`./data/databases/antinuke.db`, WAL on by default);
in cloud mode each schema is a Postgres logical schema (`antinuke.settings`). Your code never sees
the difference.

> `statementTimeout` (default 30s, `0` disables) caps a single operation. Without one, a query that
> never answers holds a pool connection for the life of the process — `max` of those and every later
> read blocks with no error at all.

The drivers are **optional peer dependencies**, loaded lazily the first time you `connect()` in that
mode — a SQLite-only app never pulls in `pg`, and vice versa. Install just the one you run:

```bash
npm install better-sqlite3 # local mode
npm install pg # cloud mode
```

<sub>Prefer the branded name? `@creative-softworks/sql-switch` re-exports this package unchanged.</sub>

## How it works

<p align="center">
<img src="./assets/readme/architecture.svg" width="100%" alt="Fluent-API calls pass through a write collector into a lazily loaded, mode-gated driver targeting either local SQLite files or PostgreSQL schemas; engineSwap migrates data between the two, chunked and resumable">
</p>

- **Write collector** — buffers writes in RAM and flushes them in bulk on an interval, collapsing
repeated writes to the same key inside the window. `.force()` bypasses it for an immediate write.
- **Circuit breaker** — caps pending writes at 5000 keys and trips to read-only on a Postgres
outage instead of crashing, then heals itself once the database answers again.
- **Exit flush** — `SIGINT`, `SIGTERM` and `beforeExit` all drain the buffer on the way out; the
library never calls `process.exit()` for you.
- **Engine swap** — moves data both directions, a chunk at a time, journalled so an interrupted run
resumes deterministically.

## Enumerate, scan & convenience helpers

Expand Down Expand Up @@ -106,7 +140,8 @@ Defaults are chosen so nothing is silently lost. All of it is configurable.

```ts
await db.connect({
db: { mode: 'cloud', connectionString: process.env.DATABASE_URL },
db: 'cloud',
cloud: { connectionString: process.env.DATABASE_URL },
collector: {
time: 3000, // flush interval
autoRecover: true, // breaker heals itself after an outage
Expand All @@ -130,16 +165,13 @@ await db.connect({

## Engine swap

Move your data between engines either from the terminal or from code.
Move your data between engines from the terminal or from code.

### CLI

```bash
# local SQLite → production PostgreSQL
npm run db:engine-swap -- --up

# production PostgreSQL → local SQLite
npm run db:engine-swap -- --down
npm run db:engine-swap -- --up # local SQLite → production PostgreSQL
npm run db:engine-swap -- --down # production PostgreSQL → local SQLite
```

| Flag | Description |
Expand All @@ -152,9 +184,8 @@ npm run db:engine-swap -- --down

### From code

Same migration, no terminal. Anything you leave out is filled in — `dataDir` defaults to
`./data/databases`, `connectionString` to `process.env.DATABASE_URL`, and missing schemas,
tables and directories are created on the target side.
Anything you leave out is filled in — `dataDir` defaults to `./data/databases`, `connectionString`
to `process.env.DATABASE_URL`, and missing schemas, tables and directories are created on the target.

```ts
import { engineSwap } from 'sql-switch';
Expand All @@ -168,9 +199,8 @@ const result = await engineSwap({
console.log(`${result.totalRows} rows across ${result.tables.length} tables`);
```

Or swap a live DAL and keep using the same object. Pending writes are flushed and the open
handles closed first, then it reconnects on the target engine with your existing collector
settings:
Or swap a live DAL and keep using the same object — pending writes are flushed and handles closed
first, then it reconnects on the target engine with your existing collector settings:

```ts
await db.swapEngine({ direction: 'up', onConflict: 'overwrite' });
Expand All @@ -190,7 +220,7 @@ await engineSwap({

### What the migration guarantees

| | |
| Guarantee | Detail |
|---|---|
| Memory | Rows stream a chunk at a time in both directions — peak memory is one chunk, not one table. |
| Atomicity | Each table moves in its own transaction going up; going down the file is built as `.tmp` and renamed into place. |
Expand Down Expand Up @@ -225,15 +255,16 @@ if (result.skippedNames.length) console.warn('left alone:', result.skippedNames)
| `table.startsWith(prefix)` | Sugar for `.entries({ prefix })`. Prefix is bound, never a pattern. |
| `table.count(opts?)` | Row count, done in the DB (rows never materialized). |
| `table.deleteAll(opts?)` | Delete every key (or just those under a prefix). |
| `db.reconnect(target?)` | Re-open the current engine (restart / recover a wedged connection), or repoint to the other declared engine (`'local'`/`'cloud'`) without moving data. Flushes first; fail-safe. |
| `db.swapEngine(options)` | Migrate to the other engine & reconnect on it. |
| `db.pendingWrites` | Number of writes currently buffered in the collector. |
| `db.close()` | Flush pending writes and close all connections. |
| `engineSwap(options)` | Standalone engine swap, no DAL instance needed. |

## Limits

The library is built so the only hard wall you hit is the database running out of storage. The few
non-storage constraints below are deliberate, so they're documented rather than left as surprises.
The only hard wall you hit is the database running out of storage. The few non-storage constraints
below are deliberate, so they're documented rather than left as surprises.

| Limit | Detail |
|-------|--------|
Expand All @@ -258,3 +289,8 @@ npm run docs:serve # serve /docs on http://localhost:3000
## Requirements

- Node.js >= 22.0.0 (tested on the active LTS / current lines, 22 and 24)

## License

[MIT](./LICENSE)

75 changes: 75 additions & 0 deletions assets/readme/architecture.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading