Draft: de-bespoke build by michalhosna · Pull Request #519 · openmoq/moqx

michalhosna · 2026-07-27T15:31:55Z

Very much a draft that needs more passes.

Goal

Make the repo easier to understand and work with for newcomers.

Make the build system less bespoke — lean more heavily on CMake, so overrides
and extension points are predictable.
Get rid of submodules.
- Submodules are a pain to manage.
- Yes, I know about submodule.recurse=true. Still a pain.
- There is a reason package managers exist and people use them instead of git
  submodules.
- We support prebuilts — a build that never touches that subtree still drags
  the source along. Optional submodule checkout is worse.
Make the scripts easier to use and understand. Less walls of text.

Breaking changes

Breaks existing scripts, and possibly the compile cache (ccache still works).
The goal here is cleanup, so breaking backwards compatibility seems ok-ish.

This change is

Inert until CMakeLists.txt includes it. The rev-to-release resolution and its consistency checks are documented in the file header.

- The from-source alternative to the prebuilt, for revs and platforms with no published tarball and for instrumented sanitizer builds. - moqx itself is not built here; only the origin of the moxygen prefix differs between the two modes. - Inert until scripts/configure.sh --mode from-source drives it.

- Pins live in cmake/dependencies.cmake, and moxygen is consumed as an installed CMake package: either the published prebuilt or a superbuild prefix. - binaryDir is build/<presetName> so profiles coexist. - No .a preference in CMAKE_FIND_LIBRARY_SUFFIXES. folly-targets.cmake hardcodes libglog.so, so an .a-preferring Glog resolution loads both copies and glog's double flag registration crashes under ASan.

Two equal dependency modes, the pin table, and the scripts that drive them.

- With CPM the build dir's CMakeCache is the only state, so the ~500-line dispatcher and its marker files go. - --mode is deliberately explicit: a default would silently pick between an hours-long stack build and an uninstrumented prebuilt. - Job count is -j, then MOQX_BUILD_JOBS, then a RAM-derated default for sanitizer profiles; an explicit value is never clamped (distcc).

scripts/dev/lint.sh needs it and Ninja does not write it unasked.

- Only the trilogy, install-system-deps.sh, and moqx-run.sh are daily entry points; the rest move next to their audience. - Not a pure move: dev/sync-relay.sh is also repointed off the deleted submodule onto the pin.

- Dependency caches are content-keyed on the pins and restored by prefix: a bump would otherwise cold-download the full set in ~11 concurrent jobs. - Every build path installs system libraries through install-system-deps.sh. - version-release resolves the release from MOXYGEN_REV, not moxygen's newest tag, which would strand the release as a draft.

- Dead weight once CPM supplies both. - Split from the CPM wiring so the license-header churn stays a separate, mechanical diff.

MOQX_BUILD_TESTS defaults ON, so the release built every test executable and never ran ctest.

MOQBIN follows whatever moxygen the build resolved: ctest passes it per test, and scripts outside ctest source the moqx-tools.env the configure writes.

Both are build-level requirements rather than per-preset choices: a bare `cmake -S .` needs them too.

Both default to ~/.cache/moqx/cpm, so a raw `cmake --preset` build reuses the cached moxygen/catapult clones instead of re-fetching into its own _deps.

Every build and test passes an explicit build/<profile>.

- A missing package surfaced deep inside the folly stack, not as the install-system-deps.sh hint. - One Boost component list, so the superbuild's static-vs-shared probe and the check cannot name different sets.

- README drops the submodule bootstrap and `build.sh setup` for the trilogy. - ci-architecture documents the shared ~/.cache/moqx cache and how it is keyed. - release replaces the `.moxygen-release` tag file with the MOXYGEN_REV pin, so a release/* branch must freeze on a v*-tagged rev. - tools/.hidden only kept tools/ in git before it had content.

- CI runs scripts/dev/format.sh; lint.sh is a manual entry point. - The targets' tools/*.cc globs never matched anything.

- The static libs are not consumable without installed headers or an install(EXPORT). - They bloat the docker image and the release tarballs.

Tests gate on MOQX_BUILD_TESTS, and no subproject reads BUILD_TESTING.

- Nothing built or registered changes. - The load test stays unregistered: it needs a live relay. - The shell-test blocks stay explicit: their properties vary.

One SanitizerFlags.cmake shared with the superbuild, and a moqx_warnings INTERFACE target linked PRIVATE so nothing propagates to consumers.

- About two thirds of compile_commands.json is CPM-fetched dependency code. - Making .clang-tidy load surfaces ~18k warnings; that is #518.

Each restated its own code and ran past the 3-line cap.

michalhosna · 2026-07-28T11:31:23Z

@afrind @gmarzot What do you think about this proposal?

At least high-level, switch to CPM, better support for custom CMake profiles etc.

gmarzot · 2026-07-28T12:06:26Z

@afrind @gmarzot What do you think about this proposal?

At least high-level, switch to CPM, better support for custom CMake profiles etc.

I see the benefits of some simplification, and improved organization. I neither love nor hate the submodule approach. I think a full walk-thru would help and being clear on some key workflows. one nagging problem that is handled to some extent in the current model (not perfect) is pr validation against the dependency profile, including our submodules, that exist in main. the other workflow difficulty is the case where you have changes in moqx that depend some unmerged work in moxygen . oh also note now openmoq/moxygen maintains artifacts for snapshot-latest and full tagged releases only... we may want to reexamine that approach too.

michalhosna · 2026-07-29T18:32:57Z

+# on the next reconfigure of an existing build dir. A cached pin would silently
+# shadow the file's value.
+set(MOXYGEN_REPOSITORY "openmoq/moxygen")
+set(MOXYGEN_REV "a1c42e1a0e0305484eb6cbcfd8c228a4685cade1")


I would point to this as one of the motivating fixes.

This is now just a plain text file. Rebase, merging, diffing now works as with any ordinary file.

Conceptually, submodules work as this simple pointer, but them being special makes them more difficult to work with. Because

Commands don't recourse by default.

The submodule state lives in multiple-places that can desync (.gitmodules, .git/config, and the actual working tree)

Different tools shows different diffs.

michalhosna added 23 commits July 28, 2026 09:42

build: fetch the published moxygen prebuilt for a pinned rev

8d04ad2

Inert until CMakeLists.txt includes it. The rev-to-release resolution and its consistency checks are documented in the file header.

docs: rewrite BUILD.md for the CPM flow

cbe8c5b

Two equal dependency modes, the pin table, and the scripts that drive them.

build: export compile_commands.json

6505fd2

scripts/dev/lint.sh needs it and Ninja does not write it unasked.

scripts: group by audience into dev/, perf/, and docker/

a43971b

- Only the trilogy, install-system-deps.sh, and moqx-run.sh are daily entry points; the rest move next to their audience. - Not a pure move: dev/sync-relay.sh is also repointed off the deleted submodule onto the pin.

submodules: remove moxygen and catapult; repoint license headers

1d821e8

- Dead weight once CPM supplies both. - Split from the CPM wiring so the license-header churn stays a separate, mechanical diff.

ci: stop compiling tests in the version-release job

4429178

MOQX_BUILD_TESTS defaults ON, so the release built every test executable and never ran ctest.

test: resolve moxygen tool paths from the build, not .scratch

c3f777f

MOQBIN follows whatever moxygen the build resolved: ctest passes it per test, and scripts outside ctest source the moqx-tools.env the configure writes.

build: move the module path and policy floor out of the presets

f426625

Both are build-level requirements rather than per-preset choices: a bare `cmake -S .` needs them too.

build: share one CPM source cache with the superbuild

299888c

Both default to ~/.cache/moqx/cpm, so a raw `cmake --preset` build reuses the cached moxygen/catapult clones instead of re-fetching into its own _deps.

presets: drop the unused build and test presets

a13e9be

Every build and test passes an explicit build/<profile>.

superbuild: fail fast on missing system -dev packages

fad483a

- A missing package surfaced deep inside the folly stack, not as the install-system-deps.sh hint. - One Boost component list, so the superbuild's static-vs-shared probe and the check cannot name different sets.

build: delete the unused format/lint custom targets

d6a5761

- CI runs scripts/dev/format.sh; lint.sh is a manual entry point. - The targets' tools/*.cc globs never matched anything.

build: install only the executables

e99bef7

- The static libs are not consumable without installed headers or an install(EXPORT). - They bloat the docker image and the release tarballs.

docker: drop the no-op BUILD_TESTING flag

3620a7c

Tests gate on MOQX_BUILD_TESTS, and no subproject reads BUILD_TESTING.

test: fold the repeated gtest boilerplate into moqx_add_gtest()

ee8be0f

- Nothing built or registered changes. - The load test stays unregistered: it needs a live relay. - The shell-test blocks stay explicit: their properties vary.

build: dedupe the sanitizer flags and the warning options

f3428e5

One SanitizerFlags.cmake shared with the superbuild, and a moqx_warnings INTERFACE target linked PRIVATE so nothing propagates to consumers.

dev: scope lint.sh to moqx's own translation units

18cf4c4

- About two thirds of compile_commands.json is CPM-fetched dependency code. - Making .clang-tidy load surfaces ~18k warnings; that is #518.

build: compress three overgrown CMakeLists.txt comments

c7bb4f2

Each restated its own code and ran past the 3-line cap.

michalhosna force-pushed the build-system-cpm branch from 0ec8de8 to c7bb4f2 Compare July 28, 2026 10:49

michalhosna requested review from afrind and gmarzot and removed request for gmarzot July 28, 2026 11:30

michalhosna commented Jul 29, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Draft: de-bespoke build - #519

Draft: de-bespoke build#519
michalhosna wants to merge 23 commits into
mainfrom
build-system-cpm

michalhosna commented Jul 27, 2026 •

edited

Loading

Uh oh!

michalhosna commented Jul 28, 2026

Uh oh!

gmarzot commented Jul 28, 2026

Uh oh!

michalhosna Jul 29, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

michalhosna commented Jul 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Goal

Breaking changes

Uh oh!

michalhosna commented Jul 28, 2026

Uh oh!

gmarzot commented Jul 28, 2026

Uh oh!

michalhosna Jul 29, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

michalhosna commented Jul 27, 2026 •

edited

Loading