Skip to content
Draft
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 70 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
# AGENTS.md

## Cursor Cloud specific instructions

3FS (Fire-Flyer File System) is a C++/Rust distributed filesystem. Components: `mgmtd`
(cluster management), `meta` (metadata service, backed by FoundationDB), `storage`
(CRAQ chain replication), `monitor_collector`, `admin_cli`, and `hf3fs_fuse_main`
(FUSE client). Build outputs land in `build/bin/`. Standard build/deploy steps live in
`README.md` and `deploy/README.md`; only the non-obvious, environment-specific caveats
are captured here.

### Toolchain pinning (important, non-obvious)
- This VM is Ubuntu 24.04, but the project targets **clang-14 + libstdc++-12** (as in CI,
which runs on Ubuntu 22.04). clang-14 auto-selects the newest GCC dir it finds, which
here is an **incomplete gcc-14** (`/usr/lib/gcc/x86_64-linux-gnu/14` has `crtbegin.o`
but no `libstdc++.so`/headers). So clang-14 must be **pinned to gcc-12** or the build
fails with `'algorithm' file not found` / `cannot find -lstdc++`. The pin flags are
already baked into the configure command below.
- `cc`/`c++` alternatives are set to **gcc-12/g++** (matching CI's default). Do not point
them back at clang, or the Rust C++ build scripts and linking break.
- Rust must be **1.85.0** (`rustup default` is already 1.85.0). Do NOT use newer stable
(>=1.90): it defaults to `rust-lld`, which fails to link the C++ deps
(`unable to find library -lstdc++`).

### Build commands
Run from the repo root (`/workspace`):
```bash
# Rust crates (uses gcc via the cc/c++ alternatives)
cargo build --release

# C++ (clang-14 pinned to gcc-12 libstdc++; SHUFFLE_METHOD is mandatory)
GCC12=/usr/lib/gcc/x86_64-linux-gnu/12
cmake -S . -B build \
-DCMAKE_CXX_COMPILER=clang++-14 -DCMAKE_C_COMPILER=clang-14 \
-DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_EXPORT_COMPILE_COMMANDS=ON \
-DSHUFFLE_METHOD=g++11 \
-DCMAKE_CXX_FLAGS="-isystem /usr/include/c++/12 -isystem /usr/include/x86_64-linux-gnu/c++/12 -B$GCC12" \
-DCMAKE_EXE_LINKER_FLAGS="-L$GCC12" -DCMAKE_SHARED_LINKER_FLAGS="-L$GCC12" -DCMAKE_MODULE_LINKER_FLAGS="-L$GCC12"
cmake --build build -j "$(nproc)" -- -k # -k: keep going (see broken test target below)
```
- Known broken target: `tests/common/utils/TestShuffle.cc` fails to compile
(`std::basic_ifstream` undefined — the file uses `std::ifstream` without
`#include <fstream>`). This is a latent repo issue, unrelated to the toolchain (fails
under both libstdc++-12 and -13). Build with `-- -k` so every other target
(all service/client binaries + the other test executables) still builds.

### Lint
From `build/`: `make check-format` (clang-format-14 dry-run) and `make clang-tidy`.
Rust: `cargo fmt` / `cargo clippy`.

### Tests
Test binaries are in `build/tests/` (gtest). Run them directly, e.g.
`./build/tests/test_meta`, `test_mgmtd`, `test_kv`, `test_client`.

### RUNTIME LIMITATION — no RDMA in this VM
This microVM has **no RDMA kernel subsystem** (no `/sys/class/infiniband`, no loadable
modules, `ibv_get_device_list` returns ENOSYS). Every service/client binary
(`mgmtd_main`, `meta_main`, `storage_main`, `hf3fs_fuse_main`, `admin_cli`,
`simple_example_main`) calls `net::IBManager::start(...)` at startup and treats failure
as **FATAL**, so none of them can run here. Consequently the full local cluster + FUSE
harness `tests/fuse/run.sh` cannot run (it also assumes `RDMA://` transport). Soft-RoCE
(`rdma_rxe`) is not possible because the kernel ships no modules.
- Functional verification is therefore done via the gtest suites that use `TCP`/`LOCAL`
transport and in-process fakes: `test_kv`, `test_mgmtd`, `test_meta`, and `test_client`
all pass (the one `test_client` failure, `testRetryUnknownAddrs`, is a TCP-only
corner case). Test suites under `tests/storage/`, `tests/migration/`, and
`tests/common/net/ib/` call `IBManager::start` and will fail for the same RDMA reason.
- A local single-node **FoundationDB** server is installed and running
(`/etc/foundationdb/fdb.cluster`); `fdbcli --exec "status minimal"` should report the
DB available.