PledgeDB

Embedded SQL database with vector search — Rust + WASM. The "SQLite of AI."

PledgeDB is an embedded SQL database built in Rust with first-class vector search support. It runs natively on servers, in the browser via WebAssembly, and on mobile devices. No server required.

Why PledgeDB?

Every AI application needs three things:

1. Structured data storage (SQL tables, queries, joins)
2. Vector search (semantic similarity for RAG, recommendations, search)
3. Local/edge deployment (privacy, latency, offline-first)

Today, you need three separate tools for this: SQLite + a vector DB (Pinecone/Qdrant) + a sync layer. PledgeDB replaces all three with a single embedded database.

Features

• Full SQL — CREATE, INSERT, SELECT, UPDATE, DELETE, DROP, WHERE, ORDER BY, LIMIT, OFFSET, DISTINCT, HAVING
• JOINs — INNER (hash join for equi-joins, nested loop for others), LEFT, RIGHT joins with ON conditions
• Aggregations — COUNT, SUM, AVG, MIN, MAX with GROUP BY
• Advanced SQL — CASE expressions, UNION/INTERSECT/EXCEPT, CTEs (WITH), derived tables, scalar/correlated subqueries (IN, EXISTS), window functions (ROW_NUMBER, RANK, DENSE_RANK, LAG, LEAD), EXPLAIN
• Transactions — BEGIN, COMMIT, ROLLBACK with undo log
• Crash recovery — WAL (write-ahead log) with CRC32 checksums and automatic replay on reopen
• Thread-safe — Send + Sync Database with RwLock for concurrent reads via ReadonlyExecutor
• MVCC — Snapshot isolation for concurrent transactions
• Encryption at rest — AES-256-GCM page-level encryption with ring
• Replication — WAL shipping primary/replica with idempotent apply
• Vector search — HNSW index for approximate nearest neighbor search, brute-force fallback
• Distance metrics — L2 (Euclidean), Cosine, Dot product with SIMD acceleration
• Full-text search — BM25 scoring with inverted index and MATCH() function
• Hybrid search — Combined vector + FTS score fusion in a single query
• Query optimizer — Logical plan IR, statistics, cost model, predicate pushdown, join reordering
• Prepared statements — ? and $N parameter binding with cached plans
• Secondary indexes — BTree-backed secondary indexes with automatic maintenance
• Bloom filters — Internal B-tree node bloom filters for faster point lookups
• Backup/restore — backup_to() and restore_from() for database snapshots
• Table compaction — Rebuild B-trees to reclaim space from deleted rows
• Group commit — Batch multiple transactions into a single fsync for high-throughput workloads
• WASM-native — Purpose-built for browser, not C-compiled-through-Emscripten
• Multi-platform — Rust, Python, JavaScript/WASM, React Native, Cloudflare Workers
• Browser persistence — OPFS + IndexedDB backends
• Rust core — Memory safe, no segfaults, no undefined behavior

Quick Start

Rust

use pledgedb_core::Database;

let db = Database::open_or_create("mydb.pldb")?;

// Create a table with a vector column
db.execute("CREATE TABLE documents (
    id INTEGER PRIMARY KEY,
    title TEXT NOT NULL,
    embedding VECTOR(128)
)")?;

// Insert data
db.execute("INSERT INTO documents (title, embedding) VALUES ('Hello', '[0.1, 0.2, ...]')")?;

// Query with JOINs and aggregations
db.execute("CREATE TABLE tags (id INTEGER PRIMARY KEY, doc_id INTEGER, name TEXT)")?;
db.execute("INSERT INTO tags (doc_id, name) VALUES (1, 'rust')")?;

let result = db.execute("
    SELECT d.title, COUNT(t.id) as tag_count
    FROM documents d
    LEFT JOIN tags t ON d.id = t.doc_id
    GROUP BY d.title
")?;

// Transactions with crash recovery
db.enable_wal("mydb.pldb")?;
db.execute("BEGIN")?;
db.execute("INSERT INTO documents (title) VALUES ('Transaction safe')")?;
db.execute("COMMIT")?;
// Data survives crashes — WAL replays committed transactions on reopen

CLI

# Initialize a database
pledgedb init --file mydb.pldb

# Start interactive shell
pledgedb shell --file mydb.pldb

# Execute a statement
pledgedb exec --file mydb.pldb "SELECT * FROM users"

# List tables
pledgedb tables --file mydb.pldb

JavaScript (WASM)

import { PledgeDB } from "pledgedb-wasm";

const db = new PledgeDB();
await db.init();
await db.exec("CREATE TABLE docs (id INTEGER PRIMARY KEY, vec VECTOR(128))");
const result = await db.exec("SELECT * FROM docs");

SQL + Vector Extensions

-- Standard SQL
CREATE TABLE articles (
    id INTEGER PRIMARY KEY,
    title TEXT NOT NULL,
    content TEXT,
    embedding VECTOR(384)
);

-- Insert with vector
INSERT INTO articles (title, content, embedding)
VALUES ('Rust Guide', 'Learn Rust...', '[0.1, 0.2, ...]');

-- Vector similarity search (find 5 most similar articles)
SELECT * FROM articles
ORDER BY embedding <-> '[0.15, 0.25, ...]'
LIMIT 5;

-- Create HNSW vector index
CREATE VECTOR INDEX idx_articles_embedding
ON articles(embedding) USING HNSW (m=16, ef_construction=200);

Architecture

pledgedb/
├── crates/
│   ├── pledgedb-core/           # SQL parser, storage engine, WAL, query executor, optimizer, FTS, MVCC, encryption
│   ├── pledgedb-vector/         # HNSW index, distance metrics (SIMD), quantization
│   ├── pledgedb-wasm/           # WASM bindings (browser/edge) with OPFS/IndexedDB persistence
│   ├── pledgedb-python/         # Python bindings (PyO3)
│   ├── pledgedb-react-native/   # React Native bindings (AsyncStorage persistence)
│   ├── pledgedb-workers/        # Cloudflare Workers bindings (KV/R2 persistence)
│   ├── pledgedb-cli/            # Command-line tool with interactive shell
│   ├── pledgedb-bench/          # Criterion benchmarks (insert, scan, select, vector search, WAL recovery)
│   └── pledgedb-fuzz/           # libFuzzer fuzz targets (parser, storage, WAL recovery, vector search)
├── docs/                    # Documentation (architecture, SQL reference, getting started, etc.)
├── Cargo.toml               # Workspace config
├── COMPARISON.md            # Competitor analysis
└── README.md                # This file

Core Crates

Crate	Description
pledgedb-core	SQL parser (sqlparser-rs), page-based storage with B-tree splitting, WAL crash recovery, query executor with JOINs/aggregations/window functions/subqueries, query optimizer, FTS (BM25), MVCC, encryption at rest, bloom filters, secondary indexes
pledgedb-vector	HNSW index, SIMD-accelerated distance metrics (L2, Cosine, Dot), scalar/binary quantization
pledgedb-wasm	WebAssembly bindings with OPFS/IndexedDB persistence, async save/load
pledgedb-python	Python bindings via PyO3 (in-memory + file-based)
pledgedb-react-native	React Native bindings with AsyncStorage persistence
pledgedb-workers	Cloudflare Workers bindings with KV/R2 persistence
pledgedb-cli	Interactive SQL shell with dot commands, CSV import, JSON/table output modes
pledgedb-bench	Criterion benchmarks for insert, scan, select, vector search, WAL crash recovery
pledgedb-fuzz	libFuzzer fuzz targets for parser, storage, WAL recovery, vector search

Roadmap

Phase 1 — Core Database (Current)

• SQL parser (CREATE, INSERT, SELECT, UPDATE, DELETE, DROP)
• Page-based storage engine with B-tree
• Query executor with WHERE, ORDER BY, LIMIT, OFFSET
• HNSW vector index with SIMD distance metrics
• CLI tool with interactive shell
• WAL (write-ahead log) for crash recovery
• JOINs (INNER, LEFT, RIGHT) with ON conditions
• Aggregations (COUNT, SUM, AVG, MIN, MAX, GROUP BY)
• Transactions (BEGIN, COMMIT, ROLLBACK) with undo log
• MVCC concurrency control

Phase 2 — AI Extensions

• Vector search in SQL (ORDER BY embedding <-> query LIMIT k)
• CREATE VECTOR INDEX syntax
• Full-text search (BM25)
• Hybrid search (vector + FTS + filters)
• Quantization (Int8, Binary)

Phase 3 — WASM & Multi-Platform

• WASM build with OPFS persistence (IndexedDB fallback)
• JavaScript/TypeScript bindings
• Python bindings (PyO3)
• React Native bindings (AsyncStorage persistence)
• Cloudflare Workers support (KV/R2 persistence)

Phase 4 — Production

• Query optimizer (cost-based — plan IR, statistics, predicate pushdown, join reordering)
• Prepared statements (? and $N parameter binding)
• Thread-safe Database (Send + Sync, RwLock, execute(&self))
• Encryption at rest (AES-256-GCM with ring)
• Replication (WAL shipping primary/replica)
• Backup/restore
• Table compaction
• Group commit (SyncMode::GroupCommit)
• Secondary indexes (BTree-backed)
• Bloom filters for B-tree internal nodes
• Page checksums (CRC32)
• LRU page cache with eviction
• Hash join for INNER equi-joins
• Connection pooling
• Schema migrations
• Page-level locking

License

MIT