Skip to content

Setup C++ Infra in GiGL#558

Merged
mkolodner-sc merged 116 commits intomainfrom
mkolodner-sc/cpp-infrastructure
Apr 29, 2026
Merged

Setup C++ Infra in GiGL#558
mkolodner-sc merged 116 commits intomainfrom
mkolodner-sc/cpp-infrastructure

Conversation

@mkolodner-sc
Copy link
Copy Markdown
Collaborator

@mkolodner-sc mkolodner-sc commented Mar 25, 2026

Scope of work done

C++ Infrastructure for GiGL

This PR adds the foundational C++ infrastructure to GiGL — everything needed to write, build, lint, and test C++ extensions alongside the existing Python codebase, and the release pipeline to ship them.

Two-wheel model

GiGL is now distributed as two wheels installed together:

  • gigl — pure Python (same wheel for CPU and CUDA users)
  • gigl-core — compiled C++/CUDA pybind11 extensions, ABI-bound to the torch variant. Currently contains no C++ sources; the package builds an empty wheel now and gets populated as extensions are added.

Both wheels are versioned and released together. gigl-core is a dependency of gigl and is resolved automatically from the same GCP registry — no separate install step for users.

Build system

gigl-core/ is a standalone scikit-build-core + CMake package. CMakeLists.txt auto-discovers all csrc/python_*.cpp (and python_*.cu) files as pybind11 extension modules — adding a new extension requires only dropping source files, no CMake edits. pybind11 is fetched via FetchContent so it does not need to be pre-installed. Torch is located by asking the target Python directly (torch.utils.cmake_prefix_path).

no-build-isolation-package = ["gigl-core"] in pyproject.toml disables build isolation for gigl-core so cmake can find torch (not on PyPI) from the ambient venv when real extensions are added. The build-backend dependency group provides scikit-build-core as the PEP 517 build backend in both dev and Docker environments.

Release pipeline

release.yml builds and publishes both wheels via a matrix build — one CPU runner, one GPU runner — to two GCP Artifact Registry repos (gigl for CPU, gigl-cu128 for CUDA 12.8). RELEASING.md documents the two-workflow release process: Create Release (version bump, branch, tag, PR) followed by Release GiGL (wheel build and publish).

C++ tooling

  • Formatting.clang-format establishes style conventions. make format_cpp formats in-place.
  • Linting.clang-tidy config. make check_lint_cpp runs clangd --check in parallel (faster than clang-tidy due to preamble caching). make fix_lint_cpp applies auto-fixable violations via clang-tidy --fix (intentionally separate from make format since it rewrites logic, not just style).
  • Dependency installationrequirements/install_cpp_deps.sh installs clang-format-15, clang-tidy-15, clangd-15, clang++-15, and libstdc++-12-dev on Linux. (clang++-15 requires libstdc++-12-dev on Ubuntu 22.04 to find standard headers.)
  • Style guidedocs/cpp_style_guide.md documents formatting rules, linting rationale, and disabled checks.

C++ unit tests

gigl-core/tests/ with GoogleTest (via FetchContent). Any *_test.cpp or *_test.cu file is automatically discovered and registered — no CMake edits needed to add a test. Run via make unit_test_cpp; included in make unit_test. A stamp file (.cache/cpp_tests/.configured) avoids re-running cmake configure when nothing changed; CONFIGURE_DEPENDS auto-invalidates when source files are added or removed.

Where is the documentation for this feature?: N/A

Did you add automated tests or write a test plan?

Updated Changelog.md? NO

Ready for code review?: NO

@mkolodner-sc mkolodner-sc changed the title [WIP] Setup C++ Infra Setup C++ Infra in GiGL Apr 1, 2026
Comment thread gigl/scripts/post_install.py
@github-actions
Copy link
Copy Markdown
Contributor

GiGL Automation

@ 23:19:08UTC : Starting to build base images for CUDA and CPU.
This may take a while, please be patient.
The images will be pushed to the GCP Artifact Registry.
Once done, the workflow will update the gigl/dep_vars.env file with the new image names.

@github-actions
Copy link
Copy Markdown
Contributor

GiGL Automation

@ 23:22:33UTC : Starting to build base images for CUDA and CPU.
This may take a while, please be patient.
The images will be pushed to the GCP Artifact Registry.
Once done, the workflow will update the gigl/dep_vars.env file with the new image names.

@github-actions
Copy link
Copy Markdown
Contributor

GiGL Automation

@ 23:32:11UTC : Built and pushed new images:

  • CUDA base image: us-central1-docker.pkg.dev/external-snap-ci-github-gigl/public-gigl/gigl-cuda-base:b598f3d72eee47f5513dcb39460944459a0a012f.108.1
  • CPU base image: us-central1-docker.pkg.dev/external-snap-ci-github-gigl/public-gigl/gigl-cpu-base:b598f3d72eee47f5513dcb39460944459a0a012f.108.1
  • Dataflow base image: us-central1-docker.pkg.dev/external-snap-ci-github-gigl/public-gigl/gigl-dataflow-base:b598f3d72eee47f5513dcb39460944459a0a012f.108.1
  • Builder image: us-central1-docker.pkg.dev/external-snap-ci-github-gigl/gigl-base-images/gigl-builder:b598f3d72eee47f5513dcb39460944459a0a012f.108.1

Updated gigl/dep_vars.env with new image names.
Updated .github/cloud_builder/run_command_on_pr_cloud_build.yaml with new builder image.

Copy link
Copy Markdown
Collaborator

@svij-sc svij-sc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!
Once the subsequent PR is in, lets do a formal release and test everything

Comment thread .github/workflows/release.yml Outdated
Comment thread gigl-core/pyproject.toml
Comment thread .github/workflows/on-pr-merge.yml
@mkolodner-sc mkolodner-sc marked this pull request as ready for review April 28, 2026 20:49
@mkolodner-sc mkolodner-sc enabled auto-merge April 28, 2026 20:49
@mkolodner-sc mkolodner-sc added this pull request to the merge queue Apr 28, 2026
@github-merge-queue github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Apr 28, 2026
@mkolodner-sc mkolodner-sc added this pull request to the merge queue Apr 28, 2026
@github-merge-queue github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Apr 29, 2026
@mkolodner-sc
Copy link
Copy Markdown
Collaborator Author

/unit_test

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 29, 2026

GiGL Automation

@ 24:29:04UTC : 🔄 Python Unit Test started.

@ 01:40:23UTC : ✅ Workflow completed successfully.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 29, 2026

GiGL Automation

@ 24:29:04UTC : 🔄 Scala Unit Test started.

@ 24:36:25UTC : ✅ Workflow completed successfully.

@mkolodner-sc mkolodner-sc added this pull request to the merge queue Apr 29, 2026
Merged via the queue into main with commit 98bf640 Apr 29, 2026
7 checks passed
@mkolodner-sc mkolodner-sc deleted the mkolodner-sc/cpp-infrastructure branch April 29, 2026 01:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants