Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions docs/deployment/clustering.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,25 @@ FLO_CLUSTER_SEEDS=10.0.1.10:9000,10.0.1.11:9000
FLO_NODE_ID=1
```

### Ports

A clustered node binds an additional Raft port for inter-node traffic:

| Port | Purpose |
|------|---------|
| `listen_port` | Client wire protocol |
| `listen_port + 500` | Raft RPC between nodes (9500 by default) |
| `listen_port + 600` | Gossip, only when `gossip_port` is set |

The Raft port is bound **only when the node can actually have peers** — when
any of `[cluster] enabled = true`, `seeds`, an explicit `raft_port`, or
`replication_factor > 1` is configured. A single-node server leaves it unbound,
so it is not a port operators need to declare to an orchestrator. The startup
banner reports either the bound port or `not listening (single-node)`.

Raft and gossip ports carry unauthenticated inter-node traffic and must not be
exposed outside the cluster's private network.

### Bootstrap

The first node in a cluster bootstraps automatically when no seeds respond. Subsequent nodes join by contacting any existing member.
Expand Down
83 changes: 81 additions & 2 deletions docs/deployment/docker.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -14,18 +14,68 @@ docker pull ghcr.io/floruntime/flo:latest
```bash
docker run -d \
--name flo \
--security-opt seccomp=unconfined \
-p 9000:9000 \
-p 9001:9001 \
-p 9002:9002 \
-v flo-data:/data \
ghcr.io/floruntime/flo:latest
```

:::note
`--security-opt seccomp=unconfined` is required on Linux hosts. Flo's event
loop uses io_uring, which Docker's default seccomp profile blocks — without it
the server starts, logs `topology manifest created`, and then exits. See
[io_uring and seccomp](#io-uring-and-seccomp) for the details and for a
narrower profile.
:::

| Port | Purpose |
|------|---------|
| 9000 | Binary wire protocol (client traffic) |
| 9001 | Prometheus metrics |
| 9002 | Dashboard REST API |
| 9001 | Prometheus metrics — only when `[metrics] enabled = true` |
| 9002 | Dashboard REST API — only when `[dashboard] enabled = true` |

Flo does **not** listen on any other port in a single-node deployment. In
particular the Raft port (`listen_port + 500`, so 9500) is bound only when the
node is actually clustered — see [Clustering](/deployment/clustering).

## io_uring and seccomp

On Linux, Flo's event loop is built on io_uring. Docker's default seccomp
profile does not allow the `io_uring_setup`, `io_uring_enter` and
`io_uring_register` syscalls, so under that profile Flo cannot start:

```
INF topology manifest created: shards=1 partitions=0 path=/data
ERR io_uring is unavailable (PermissionDenied). ...
Error starting runtime: error.PermissionDenied
```

The simplest fix is to run unconfined:

```bash
--security-opt seccomp=unconfined
```

If you would rather not drop the whole profile, copy Docker's
[default profile](https://github.com/moby/moby/blob/master/profiles/seccomp/default.json)
and add the three syscalls to its allowlist:

```json
{
"names": ["io_uring_setup", "io_uring_enter", "io_uring_register"],
"action": "SCMP_ACT_ALLOW"
}
```

Then run with `--security-opt seccomp=/path/to/flo-seccomp.json`.

:::note
io_uring requires **Linux 5.1 or newer**. On older kernels Flo cannot start at
all, and no seccomp profile will help. Kubernetes, Podman and containerd apply
their own seccomp defaults — the same allowance is needed there.
:::

## Docker Compose — Single Node

Expand All @@ -34,6 +84,8 @@ version: "3.8"
services:
flo:
image: ghcr.io/floruntime/flo:latest
security_opt:
- seccomp:unconfined
ports:
- "9000:9000"
- "9001:9001"
Expand All @@ -56,6 +108,8 @@ version: "3.8"
services:
flo-1:
image: ghcr.io/floruntime/flo:latest
security_opt:
- seccomp:unconfined
hostname: flo-1
ports:
- "9000:9000"
Expand All @@ -71,6 +125,8 @@ services:

flo-2:
image: ghcr.io/floruntime/flo:latest
security_opt:
- seccomp:unconfined
hostname: flo-2
ports:
- "9010:9000"
Expand All @@ -86,6 +142,8 @@ services:

flo-3:
image: ghcr.io/floruntime/flo:latest
security_opt:
- seccomp:unconfined
hostname: flo-3
ports:
- "9020:9000"
Expand Down Expand Up @@ -122,6 +180,8 @@ All `flo.toml` settings can be overridden with `FLO_` prefixed environment varia

## Health Check

For a **liveness** probe inside the container, the CLI is enough:

```yaml
healthcheck:
test: ["CMD", "flo", "server", "status"]
Expand All @@ -130,6 +190,25 @@ healthcheck:
retries: 3
```

For an orchestrator **readiness** probe that comes from outside the container,
use `GET /health` on the dashboard port (see
[REST API](/reference/rest-api#cluster-metrics)) — and set the dashboard's bind
address, because it defaults to localhost only:

```toml
[dashboard]
enabled = true
bind = "0.0.0.0"
```

:::note
`[dashboard] bind` defaults to `127.0.0.1`. A probe originating outside the
container cannot reach a listener bound to loopback, so leaving the default in
an orchestrated deployment presents as a service that never becomes ready —
which reads like a broken health check rather than a bind address. Bind to
`0.0.0.0` only where the dashboard port is not publicly routable.
:::

## Volumes and Data Persistence

Always mount a volume at `/data` to persist data across container restarts:
Expand Down
46 changes: 40 additions & 6 deletions docs/getting-started/configuration.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -48,11 +48,28 @@ hot_buffer_capacity = 67108864 # Per-partition ring buffer size in bytes (64 M

Durability modes:

| Mode | Behaviour |
|------|-----------|
| `sync` | `fsync` after every write — strongest guarantee, lowest throughput |
| `async_flush` | Background flush every ~1 ms — default, highest throughput |
| `ephemeral` | Skip WAL entirely — for caches or temporary data |
| Mode | Behaviour | At risk on abrupt termination |
|------|-----------|-------------------------------|
| `sync` | Segment written and `fsync`ed before the write is acknowledged | Nothing |
| `async_flush` | Segments written and `fsync`ed by a background task, **at most once per second** — the default | Up to ~1 second of acknowledged writes |
| `ephemeral` | Never written to disk | Everything |

**What `async_flush` risks, concretely.** An acknowledged write lives only in
memory until the next segment flush, which runs at most once per second. If the
process is killed (`SIGKILL`, OOM kill, power loss, host failure) you can lose
up to roughly the last second of acknowledged writes. Everything older is on
disk, `fsync`ed and published by atomic rename, so the store is never left
torn or half-written — the loss is a clean tail truncation, not corruption.

A **graceful** shutdown (`SIGTERM`, `docker stop`, `flo server stop`) flushes
before exiting, so a normal restart loses nothing. The one-second window only
applies when the process dies without running its shutdown path.

:::note
This window is **not** governed by `hot_flush_seconds`. That setting controls
hot → warm tier migration, which is a storage-layout concern and does not
affect what survives a crash.
:::

### `[logging]`

Expand Down Expand Up @@ -88,9 +105,12 @@ pong_timeout_ms = 10000 # Close connection if no pong within this time
[metrics]
enabled = true
# port = 0 # 0 = auto (listen_port + 1), so 9001 by default
# bind = "0.0.0.0"
# bind = "127.0.0.1" # Localhost only by default
```

Binds only when `enabled = true`. For a Prometheus server scraping from outside
the container, set `bind = "0.0.0.0"` — the default is loopback-only.

### `[dashboard]`

```toml
Expand All @@ -101,6 +121,14 @@ enabled = true
# cors_origins = "http://localhost:5173"
```

:::note
`bind` defaults to `127.0.0.1`. In an orchestrated deployment a readiness probe
originating outside the container cannot reach a loopback listener, so the
default presents as a service that never becomes ready. Set `bind = "0.0.0.0"`
where the port is not publicly routable. See
[Docker → Health Check](/deployment/docker#health-check).
:::

### `[cluster]`

```toml
Expand All @@ -115,6 +143,12 @@ enabled = false
# heartbeat_interval_ms = 50
```

The Raft port is bound only when this node can actually have peers — that is,
when any of `enabled = true`, `seeds`, an explicit `raft_port`, or
`replication_factor > 1` is set. A plain single-node server does **not** listen
on `listen_port + 500`, so there is no undeclared port to account for. The
startup banner reports which of the two applies.

### `[cold_storage]`

```toml
Expand Down
27 changes: 26 additions & 1 deletion docs/reference/rest-api.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,32 @@ Aggregated metrics in JSON (Prometheus metrics are exposed separately on port 90

### `GET /health`

Always-public liveness check (served at the root, not under `/api/v1`).
Liveness and readiness check. Served at the root, not under `/api/v1`, and
always public — it is never gated by the admin token, so it is safe to point an
orchestrator probe at it without provisioning credentials.

```json
{ "status": "ok" }
```

A `200` means the dashboard server is up and serving. Any other status, or a
refused connection, means the node is not ready.

**Stability:** `GET /health` is a stable endpoint. The path, its public status,
and the `200` + `"status": "ok"` contract will not change within a major
version. Fields may be *added* to the object, so parse it leniently rather than
matching the body exactly — checking the HTTP status alone is the most durable
probe.

:::note
This is served on the **dashboard** port (`listen_port + 2`, so 9002 by
default), which requires `[dashboard] enabled = true`. `[dashboard] bind`
defaults to `127.0.0.1`, so a probe from outside the container needs
`bind = "0.0.0.0"` — see [Docker](/deployment/docker#health-check).

The Prometheus exporter serves its own separate `/health` on `listen_port + 1`,
which additionally reports the shard count.
:::

---

Expand Down
Loading