Skip to content

fix(cluster): honor [cluster] enabled, implement cluster status, explain io_uring (#42 items 5, 6, 7) - #47

Merged
oreofeolurin merged 2 commits into
devfrom
fix/42-cluster-transparency
Aug 29, 2026
Merged

fix(cluster): honor [cluster] enabled, implement cluster status, explain io_uring (#42 items 5, 6, 7)#47
oreofeolurin merged 2 commits into
devfrom
fix/42-cluster-transparency

Conversation

@oreofeolurin

Copy link
Copy Markdown
Contributor

Fixes items 5, 6 and the code half of 7 in #42. Docs are in floruntime/docs#9.


Item 5 — Raft bound its port with [cluster] enabled = false

Two faults here, both silent.

[cluster] enabled was parsed and thrown away:

// Note: "enabled" field is deprecated - cluster always runs
// Parse it for backwards compatibility but ignore
_ = table.getBool("enabled");

The operator set it, the parser read it, and nothing used it or said so.

The gate was structurally always-true:

if (self.config.effectiveRaftPort() != self.config.listen_port or ...)

effectiveRaftPort() derives listen_port + 500 when no port is configured, so this compares 9500 against 9000 — true for every single-node server. Raft always bound its port, while the banner printed the unset config value (Raft port: 0) next to a live listener on 9500. The reporter's exact observation.

Now enabled is honored, and the listener starts only when peers are actually possible — enabled = true, seeds, an explicit raft_port, or replication_factor > 1:

  Cluster:
    Node ID:    auto (will generate on start)
    Raft port:  not listening (single-node)
    Mode:       Single-node (no seeds)
$ lsof -nP -iTCP -sTCP:LISTEN | grep 9720
*:9720            # client
127.0.0.1:9721    # metrics
127.0.0.1:9722    # dashboard
                  # 10220 (raft) absent

This path was already supported — every consumer of shard.raft_network null-checks it and the field initialises to null. It was just unreachable.

Item 6 — flo cluster status returned "not implemented"

The opcode, wire format and the entire CLI renderer already existed. Only the server-side handler was missing, so the request fell through the dispatcher to its generic "not implemented" — for a command flo --help advertises in its own examples.

$ flo cluster status

Cluster Status
──────────────
Node ID:    flo-5b938c
Address:    127.0.0.1:9722
Role:       leader
Leader:     flo-5b938c
Term:       1
Members:    1

A single node answers as leader of a one-member cluster — the truthful answer rather than an error, which is what the report asked for. The node's cluster ID is now computed whether or not the Raft listener runs, so it's reportable single-node.

Item 7 (code half) — undiagnosable io_uring failure

A bare error.PermissionDenied that names neither io_uring nor seccomp is close to undebuggable cold. The reactor now explains it and names the fix, separating a blocked seccomp profile from a pre-5.1 kernel:

ERR io_uring is unavailable (PermissionDenied). Flo's event loop requires io_uring on Linux.
  * Under a container runtime's default seccomp profile the io_uring syscalls are blocked.
    Run with `--security-opt seccomp=unconfined`, or use a profile that allows
    io_uring_setup, io_uring_enter and io_uring_register.
  * On kernels older than 5.1, io_uring does not exist and Flo cannot start.

Tests

Unit tests pin the gate — including an explicit assertion that the old condition was true for a plain single-node config, so the bug can't come back unnoticed. E2E tests cover cluster status in table and JSON form, asserting on output rather than exit status. Both e2e tests fail against the unfixed code.

The Linux-only reactor branch can't compile on this host (macOS), and the full -Dtarget=x86_64-linux build has two pre-existing failures unrelated to this change. I type-checked it by forcing analysis of Reactor.init under -target x86_64-linux, and verified both error prongs against std's io_uring_setup mapping (EPERMPermissionDenied, ENOSYSSystemOutdated).

  • test-unit ✅ · test-integration
  • test-e2e 575/579 — the 4 failures are the kv/cluster set, confirmed failing identically on a stashed baseline. Worth noting they're unchanged by the Raft gating, which is the result that matters here.

Behaviour change

A single-node server no longer listens on listen_port + 500. Anything that assumed the port was always open (a probe, a firewall rule, a health check) will see it closed — that's the intended fix, but it is a visible change.

…ain io_uring (#42)

Issue #42 items 5, 6 and the code half of item 7.

## Item 5 — Raft bound its port with `[cluster] enabled = false`

Two separate faults, both silent.

`[cluster] enabled` was parsed and thrown away — `_ = table.getBool("enabled")`
with a comment calling the field deprecated because "cluster always runs". The
operator's setting had no effect and nothing said so.

The gate that decided whether to start the Raft network was
`effectiveRaftPort() != listen_port`, but that helper *derives* `listen_port +
500` when no port is configured, so the condition was true for every
single-node server. Raft always bound its port, while the startup banner
printed the unset config value — "Raft port: 0" — beside a live listener on
9500.

`enabled` is now honored, and the listener starts only when this node can
actually have peers: `enabled = true`, seeds, an explicit `raft_port`, or
`replication_factor > 1`. A plain single-node server leaves the port unbound.
The banner reports the port Raft will really bind, or "not listening
(single-node)".

Every consumer of `shard.raft_network` already null-checked it and the field
already initialised to null, so the unbound path was supported — just
unreachable.

## Item 6 — `flo cluster status` returned "not implemented"

The opcode, the wire format and the whole CLI renderer existed; only the
server-side handler was missing, so the request fell through the dispatcher to
its generic "not implemented" reply — for a command `flo --help` advertises in
its own examples. Registered a handler that reports this node's Raft identity,
role, term and member count. A single node answers as leader of a one-member
cluster, which is the truthful answer rather than an error.

The node's cluster ID is now computed whether or not the listener runs, so it
is reportable single-node.

## Item 7 (code half) — undiagnosable io_uring failure

Under a container runtime's default seccomp profile the io_uring syscalls are
blocked and Flo died with a bare `error.PermissionDenied` naming neither
io_uring nor seccomp. The reactor now explains the failure and names the fix,
distinguishing a blocked profile from a pre-5.1 kernel.

Docs are in floruntime/docs#9.

## Tests

Unit tests pin the gate, including an explicit assertion that the old
condition was true for a plain single-node config. E2E tests cover `cluster
status` in both table and JSON form, asserting on output rather than exit
status. Both e2e tests fail against the unfixed code.

The Linux-only reactor branch does not compile on this host, so it was
type-checked by forcing analysis under `-target x86_64-linux`. Its two error
prongs were checked against std's `io_uring_setup` mapping (EPERM →
PermissionDenied, ENOSYS → SystemOutdated).

test-unit and test-integration pass. test-e2e: 575/579, the 4 failures being
the `kv/cluster` set that fails identically on a stashed baseline — notably
unchanged by the Raft gating.
Follow-up on the same code path, both found by CI on Linux.

`Runtime.start` allocated the RaftNetwork and then called a fallible
`RaftNetwork.init` with no errdefer, so a failed bind leaked the allocation.
The DebugAllocator reported it alongside the bind failure.

The bind failure itself is already fixed by this PR's main change, and is a
sharper example of the always-true gate than the one in the description:

  `listen_port = 0` means "pick an ephemeral port", but effectiveRaftPort()
  adds 500 unconditionally and returns 500 — privileged. The old gate then
  started the listener anyway, so the two-shard runtime test tried to bind
  port 500. macOS permits bind(0.0.0.0:500) and Linux returns EACCES, so
  `Runtime: boot 2 shards and shutdown` passed locally and failed in CI.

With the new gate that config wants no listener at all, so the privileged port
is never bound. Added a unit test pinning both halves.
@oreofeolurin
oreofeolurin force-pushed the fix/42-cluster-transparency branch from 3f0222f to b09c761 Compare August 29, 2026 16:59
@oreofeolurin
oreofeolurin merged commit 0c45a8e into dev Aug 29, 2026
oreofeolurin added a commit that referenced this pull request Aug 29, 2026
…) (#53)

`listen_port = 0` means "bind an ephemeral port", so it is not a base to add an
offset to. The helpers added theirs unconditionally and produced ports 1
(metrics), 2 (dashboard), 500 (Raft) and 600 (gossip) — all privileged, all
refused on Linux without CAP_NET_BIND_SERVICE.

`Runtime: boot 2 shards and shutdown` uses `.listen_port = 0`, and both
metrics_enabled and dashboard_enabled default to true, so it tried to bind port
2 and failed. The metrics exporter hit it first but survived — it got graceful
bind-failure logging in #43; the dashboard has no such guard and failed the
whole runtime start.

An unresolved base now derives an unresolved port: the helpers return 0 when
listen_port is 0. An explicitly configured port still wins.

macOS permits bind(0.0.0.0, <low port>) where Linux does not, which is why
every local run passed. Second Linux-only failure found by the CI in #49,
after #50.

Also updates the ephemeral-port test added in #47, which asserted
effectiveRaftPort() == 500 — that pinned the buggy behaviour.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant