Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions apps/docs/astro.config.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,8 @@ export default defineConfig({
],
},
{
label: 'CLI Reference',
autogenerate: { directory: 'cli' },
label: 'Tutorials',
autogenerate: { directory: 'tutorials' },
},
{
label: 'Configuration',
Expand All @@ -54,6 +54,10 @@ export default defineConfig({
label: 'Plugins',
autogenerate: { directory: 'plugins' },
},
{
label: 'CLI Reference',
autogenerate: { directory: 'cli' },
},
],
}),
...(isDev ? [react()] : []),
Expand Down
7 changes: 1 addition & 6 deletions apps/docs/src/components/Footer.astro
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
---
/**
* Override of Starlight's Footer.
Expand Down Expand Up @@ -25,12 +25,7 @@
<path d="M8 0C3.58 0 0 3.58 0 8c0 3.54 2.29 6.53 5.47 7.59.4.07.55-.17.55-.38 0-.19-.01-.82-.01-1.49-2.01.37-2.53-.49-2.69-.94-.09-.23-.48-.94-.82-1.13-.28-.15-.68-.52-.01-.53.63-.01 1.08.58 1.23.82.72 1.21 1.87.87 2.33.66.07-.52.28-.87.51-1.07-1.78-.2-3.64-.89-3.64-3.95 0-.87.31-1.59.82-2.15-.08-.2-.36-1.02.08-2.12 0 0 .67-.21 2.2.82.64-.18 1.32-.27 2-.27.68 0 1.36.09 2 .27 1.53-1.04 2.2-.82 2.2-.82.44 1.1.16 1.92.08 2.12.51.56.82 1.27.82 2.15 0 3.07-1.87 3.75-3.65 3.95.29.25.54.73.54 1.48 0 1.07-.01 1.93-.01 2.2 0 .21.15.46.55.38A8.013 8.013 0 0 0 16 8c0-4.42-3.58-8-8-8Z" />
</svg>
</a>
<a href="#" aria-label="Discord">
<svg viewBox="0 0 24 24" fill="currentColor" aria-hidden="true">
<path d="M20.317 4.3698a19.7913 19.7913 0 00-4.8851-1.5152.0741.0741 0 00-.0785.0371c-.211.3753-.4447.8648-.6083 1.2495-1.8447-.2762-3.68-.2762-5.4868 0-.1636-.3933-.4058-.8742-.6177-1.2495a.077.077 0 00-.0785-.037 19.7363 19.7363 0 00-4.8852 1.515.0699.0699 0 00-.0321.0277C.5334 9.0458-.319 13.5799.0992 18.0578a.0824.0824 0 00.0312.0561c2.0528 1.5076 4.0413 2.4228 5.9929 3.0294a.0777.0777 0 00.0842-.0276c.4616-.6304.8731-1.2952 1.226-1.9942a.076.076 0 00-.0416-.1057c-.6528-.2476-1.2743-.5495-1.8722-.8923a.077.077 0 01-.0076-.1277c.1258-.0943.2517-.1923.3718-.2914a.0743.0743 0 01.0776-.0105c3.9278 1.7933 8.18 1.7933 12.0614 0a.0739.0739 0 01.0785.0095c.1202.099.246.1981.3728.2924a.077.077 0 01-.0066.1276 12.2986 12.2986 0 01-1.873.8914.0766.0766 0 00-.0407.1067c.3604.698.7719 1.3628 1.225 1.9932a.076.076 0 00.0842.0286c1.961-.6067 3.9495-1.5219 6.0023-3.0294a.077.077 0 00.0313-.0552c.5004-5.177-.8382-9.6739-3.5485-13.6604a.061.061 0 00-.0312-.0286zM8.02 15.3312c-1.1825 0-2.1569-1.0857-2.1569-2.419 0-1.3332.9555-2.4189 2.157-2.4189 1.2108 0 2.1757 1.0952 2.1568 2.419 0 1.3332-.9555 2.4189-2.1569 2.4189zm7.9748 0c-1.1825 0-2.1569-1.0857-2.1569-2.419 0-1.3332.9554-2.4189 2.1569-2.4189 1.2108 0 2.1757 1.0952 2.1568 2.419 0 1.3332-.946 2.4189-2.1568 2.4189Z" />
</svg>
</a>
<a href="#" aria-label="X">
<a href="https://x.com/obsessiondb" target="_blank" rel="noopener noreferrer" aria-label="X">
<svg viewBox="0 0 24 24" fill="currentColor" aria-hidden="true">
<path d="M18.901 1.153h3.68l-8.04 9.19L24 22.846h-7.406l-5.8-7.584-6.638 7.584H.474l8.6-9.83L0 1.154h7.594l5.243 6.932ZM17.61 20.644h2.039L6.486 3.24H4.298Z" />
</svg>
Expand Down
81 changes: 52 additions & 29 deletions apps/docs/src/content/docs/cli/init.md
Original file line number Diff line number Diff line change
@@ -1,68 +1,91 @@
---
title: "chkit init"
description: "Scaffold a new chkit project with config and example schema files."
description: "Scaffold a chkit project config, example schema, dependencies, and an optional database connection."
sidebar:
order: 2
---

Creates a `clickhouse.config.ts` configuration file and an example schema file in your project. If either file already exists, it is left untouched.
Scaffolds a chkit project in the current directory: writes a `clickhouse.config.ts` and an example schema file, installs dependencies if the project has none, and — on an interactive run — offers to connect a database.

## Synopsis

```
chkit init
chkit init [flags]
```

## Flags

No command-specific flags. See [global flags](/cli/overview/#global-flags).
| Flag | Type | Default | Description |
|------|------|---------|-------------|
| `--connect <choice>` | string | — | Preselect a connection path: `claim`, `account`, `clickhouse`, or `later`. Skips the interactive prompt. |
| `--email <email>` | string | — | Email for the ObsessionDB `claim` / `account` paths (implies an interactive connect step). |
| `--code <code>` | string | — | One-time email code, to verify a signup without re-prompting. |
| `--org-name <name>` | string | — | Override the auto-created ObsessionDB organization name. |
| `--yes`, `-y` | boolean | `false` | Skip the connect prompt and just write files — keeps `init` a silent scaffolder for scripts. |

See [global flags](/cli/overview/#global-flags).

## Behavior

`chkit init` writes two files relative to the current working directory:
`chkit init` runs three steps in order. Each is idempotent, so re-running on an existing project is safe.

### 1. Scaffold files

Writes two files relative to the current working directory, leaving any that already exist untouched:

1. **`clickhouse.config.ts`** — project config with sensible defaults: `schema: './src/db/schema/**/*.ts'`, `outDir: './chkit'`, `migrationsDir: './chkit/migrations'`, `metaDir: './chkit/meta'`, an empty `plugins` array, and a `clickhouse` block reading from `CLICKHOUSE_URL`, `CLICKHOUSE_USER`, `CLICKHOUSE_PASSWORD`, and `CLICKHOUSE_DB`.
2. **`src/db/schema/example.ts`** — a starter `MergeTree` table named `events` with columns `id` (`UInt64`), `source` (`String`), and `ingested_at` (`DateTime64(3)`).

1. **`clickhouse.config.ts`** — project configuration with sensible defaults:
- `schema: './src/db/schema/**/*.ts'`
- `outDir: './chkit'`
- `migrationsDir: './chkit/migrations'`
- `metaDir: './chkit/meta'`
- `plugins: []`
- `clickhouse` block reading from environment variables (`CLICKHOUSE_URL`, `CLICKHOUSE_USER`, `CLICKHOUSE_PASSWORD`, `CLICKHOUSE_DB`)
### 2. Install dependencies

2. **`src/db/schema/example.ts`** — a sample table definition using `MergeTree` engine with columns `id` (UInt64), `source` (String), and `ingested_at` (DateTime64(3)).
If `@chkit/core` does not already resolve from the project, `init` makes the project runnable: it writes a minimal `package.json` when none exists, then installs `chkit`, `@chkit/core`, and `@chkit/plugin-obsessiondb` as dev dependencies using the detected package manager (`npm`, `pnpm`, `yarn`, or `bun`; defaults to `bun`). This is why `init` works in a brand-new empty folder, not just an existing project. A failed install never aborts `init` — it prints the manual install command and continues.

The command is idempotent — running it again on an existing project does nothing.
### 3. Connect a database

On a TTY — or when `--connect` / `--email` is passed — `init` runs the ObsessionDB onboarding prompt:

```
Claim a free ObsessionDB dev instance email code, ready in seconds
I already have an ObsessionDB account log in and pick a service
I already have a ClickHouse instance connect with env vars
Configure later
```

Claiming or logging in registers the `@chkit/plugin-obsessiondb` plugin in your config and writes the selected service to `.chkit/obsessiondb.json`. See [Getting Started with ObsessionDB](/obsessiondb/getting-started/) for each path in detail, including the non-interactive and agent-friendly (`--json`) variants.

Pass `--yes` to skip this step and just write files. In a non-interactive environment (no TTY) without connect flags, `init` skips onboarding entirely — it writes files and prints static next steps (set `CLICKHOUSE_URL`, edit the schema, then `generate` and `migrate`). When an explicit connect path is requested but cannot complete — for example `--connect claim` with no email, or a wrong code — `init` exits non-zero so scripts can detect the failure.

## Examples

**Initialize a new project:**
**Scaffold and connect interactively:**

```sh
chkit init
```

Output:
**Scaffold only, no prompt (CI / scripts):**

```sh
chkit init --yes
```
Created clickhouse.config.ts
Created src/db/schema/example.ts

Next steps:
1. Set CLICKHOUSE_URL (and CLICKHOUSE_USER / CLICKHOUSE_PASSWORD / CLICKHOUSE_DB if needed).
2. Edit src/db/schema/example.ts to match your data.
3. Run: bunx chkit generate --name init
4. Run: bunx chkit migrate --apply
**Preselect the claim path (still prompts for the emailed code):**

Docs: https://chkit.obsessiondb.com/getting-started/add-to-existing-project/
```sh
chkit init --connect claim --email you@example.com
```

**Run on an existing project (no changes):**
For a fully scripted claim with no prompts, use the two-step OTP flow in [Getting Started with ObsessionDB](/obsessiondb/getting-started/#non-interactive-setup).

```sh
chkit init
# No output — both files already exist
```
## Exit codes

| Code | Meaning |
|------|---------|
| 0 | Success |
| non-zero | An explicitly requested connect path could not complete |

## Related commands

- [Getting Started: add to an existing project](/getting-started/add-to-existing-project/) — the guided first-run flow
- [Getting Started with ObsessionDB](/obsessiondb/getting-started/) — every connect path in detail
- [`chkit generate`](/cli/generate/) — generate migrations from your schema after init
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,10 @@ Add `chkit` and `@chkit/core` as dev dependencies:

Edit `src/db/schema/example.ts` to match the table you actually want before moving on.

:::note
On an interactive run, `chkit init` also offers to connect a database — including claiming a **free ObsessionDB dev instance** with a one-time email code, no local ClickHouse required. This guide wires up a connection with environment variables instead (the steps below); pass `--yes` to skip the prompt. See [Getting Started with ObsessionDB](/obsessiondb/getting-started/) for the connect paths, or the [first-schema tutorial](/tutorials/first-schema/) for an end-to-end walkthrough using the free instance.
:::

## 3. Generate the initial migration

`chkit generate` diffs your schema definitions against the previous snapshot and writes migration SQL into `chkit/migrations/`. Pass `--name` to label the migration file.
Expand Down
2 changes: 1 addition & 1 deletion apps/docs/src/content/docs/obsessiondb/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ chkit ships a dedicated integration with [ObsessionDB](https://obsessiondb.com),
- **One schema, two targets** — write `Shared*` engines once and run them against ObsessionDB as-is, or against regular ClickHouse with the `Shared` prefix stripped automatically.
- **Service selection** — list services across your organizations, pick a default per project, and override per command without touching config.
- **Remote query execution** — once a service is selected, `chkit query` and other SQL-emitting commands route through the ObsessionDB API instead of a local ClickHouse connection.
- **Remote backfills** — `chkit backfill` can submit jobs to ObsessionDB rather than streaming chunks from your machine.
- **Remote backfills** — `chkit plugin backfill` can submit jobs to ObsessionDB rather than streaming chunks from your machine.

## Install

Expand Down
9 changes: 9 additions & 0 deletions apps/docs/src/content/docs/plugins/backfill.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,19 @@
---
title: Backfill Plugin
description: Plan, execute, and monitor time-windowed backfill operations with async query submission, concurrent execution, and checkpointed progress.
sidebar:
order: 4
badge:
text: Alpha
variant: caution
---

This document covers practical usage of the optional `backfill` plugin.

:::caution[Alpha]
The `backfill` plugin is in alpha. Its API, configuration, and CLI flags may change between releases.
:::

## What it does

- Builds deterministic, immutable backfill plans that divide a time window into chunks.
Expand Down
2 changes: 2 additions & 0 deletions apps/docs/src/content/docs/plugins/codegen.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
---
title: Codegen Plugin
description: Generate TypeScript row types, optional Zod schemas, ingestion functions, and runtime migration modules from chkit schema definitions.
sidebar:
order: 2
---

This document covers practical usage of the optional `codegen` plugin.
Expand Down
2 changes: 2 additions & 0 deletions apps/docs/src/content/docs/plugins/overview.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
---
title: Plugins Overview
description: How chkit plugins work and which official plugins are available.
sidebar:
order: 1
---

Plugins extend chkit with capabilities that don't belong in the core CLI — code generation, schema introspection, data backfill, ObsessionDB integration, and anything else you want to bolt on. They're regular npm packages that you register in `clickhouse.config.ts`:
Expand Down
2 changes: 2 additions & 0 deletions apps/docs/src/content/docs/plugins/pull.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
---
title: Pull Plugin
description: Introspect live ClickHouse tables, views, and materialized views and generate chkit schema files.
sidebar:
order: 3
---

This document covers practical usage of the optional `pull` plugin.
Expand Down
29 changes: 25 additions & 4 deletions apps/docs/src/content/docs/schema/dsl-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ const users = table({
{ name: 'id', type: 'UInt64' },
{ name: 'email', type: 'String' },
],
engine: 'MergeTree()',
engine: 'MergeTree',
primaryKey: ['id'],
orderBy: ['id'],
})
Expand All @@ -67,7 +67,7 @@ const events = table({
{ name: 'received_at', type: 'DateTime64(3)', default: 'fn:now64(3)' },
{ name: 'status', type: 'String', default: 'pending', comment: 'Event processing status' },
],
engine: 'MergeTree()',
engine: 'MergeTree',
primaryKey: ['id'],
orderBy: ['org_id', 'received_at', 'id'],
partitionBy: 'toYYYYMM(received_at)',
Expand All @@ -90,7 +90,7 @@ const events = table({
| `database` | `string` | ClickHouse database name |
| `name` | `string` | Table name |
| `columns` | `ColumnDefinition[]` | Column definitions (see [Columns](#columns)) |
| `engine` | `string` | Engine clause, e.g. `'MergeTree()'`, `'ReplacingMergeTree(ver)'` |
| `engine` | `string` | Engine clause, e.g. `'MergeTree'`, `'ReplacingMergeTree(ver)'` |
| `primaryKey` | `string[]` | Primary key columns |
| `orderBy` | `string[]` | ORDER BY columns |

Expand All @@ -109,7 +109,7 @@ const events = table({
| `plugins` | `TablePlugins` | Per-table plugin configuration (see [Plugin configuration](#plugin-configuration)) |

:::note
The `engine` field accepts any string. Common engines include `MergeTree()`, `ReplacingMergeTree()`, `SummingMergeTree()`, `AggregatingMergeTree()`, and `CollapsingMergeTree(sign)`.
The `engine` field accepts any string. Common engines include `MergeTree`, `ReplacingMergeTree`, `SummingMergeTree`, `AggregatingMergeTree`, and `CollapsingMergeTree(sign)`. Empty parentheses are optional for parameterless engines (`'MergeTree'` and `'MergeTree()'` are equivalent), and chkit normalizes them when comparing schemas.
:::

:::note
Expand All @@ -130,6 +130,27 @@ Any ClickHouse type string. Parameterized types like `DateTime64(3)`, `Decimal(1

Primitive types recognized by the DSL type system: `String`, `UInt8`, `UInt16`, `UInt32`, `UInt64`, `UInt128`, `UInt256`, `Int8`, `Int16`, `Int32`, `Int64`, `Int128`, `Int256`, `Float32`, `Float64`, `Bool`, `Boolean`, `Date`, `DateTime`, `DateTime64`.

#### SQL-standard aliases

chkit passes the `type` string through to ClickHouse verbatim — it does not rewrite it. ClickHouse itself accepts standard SQL type aliases and stores them as its native types, so a table declared with aliases like `BIGINT` or `TEXT` is created successfully:

| SQL alias | ClickHouse native type |
|-----------|------------------------|
| `TINYINT` | `Int8` |
| `SMALLINT` | `Int16` |
| `INTEGER` / `INT` | `Int32` |
| `BIGINT` | `Int64` |
| `FLOAT` / `REAL` | `Float32` |
| `DOUBLE` | `Float64` |
| `TEXT` / `VARCHAR` / `CHAR` | `String` |
| `TIMESTAMP` | `DateTime` |

See the ClickHouse [data types reference](https://clickhouse.com/docs/sql-reference/data-types) for the complete alias list.

:::caution
**Prefer the native type.** chkit compares column types literally. A column declared as `BIGINT` is created as `Int64`, but [`chkit drift`](/cli/drift/) and [`chkit check`](/cli/check/) then compare your declared `BIGINT` against the live `Int64` and report a permanent `changed_column` drift — failing `chkit check --strict` on every run. The [codegen plugin](/plugins/codegen/) likewise recognizes only native names (see [Type system reference](#type-system-reference)) — an alias raises `codegen_unsupported_type`, or emits `unknown` when `failOnUnsupportedType` is `false`. Use the native ClickHouse type (`Int64`, not `BIGINT`) unless you have a specific reason not to.
:::

### `nullable` (boolean, optional)

When `true`, the column type is wrapped in `Nullable(...)` in the generated SQL.
Expand Down
Loading
Loading