Skip to content

macOS: give the shared libraries @loader_path install names (fixes cross-linking with pyscf) - #3

Open
smvinko wants to merge 1 commit into
diffqc:mainfrom
smvinko:macos-loader-path-install-names
Open

macOS: give the shared libraries @loader_path install names (fixes cross-linking with pyscf)#3
smvinko wants to merge 1 commit into
diffqc:mainfrom
smvinko:macos-loader-path-install-names

Conversation

@smvinko

@smvinko smvinko commented Jul 31, 2026

Copy link
Copy Markdown

Problem

On macOS every library in dqclibs is stamped with an @rpath/<name>.dylib
install name (CMAKE_MACOSX_RPATH ON), and pyscf ships libcgto,
libnp_helper, libcvhf and libpbc under exactly those names
.

dyld satisfies an @rpath dependency from any already-loaded image whose
install name matches — it does not consult the referring library's own
LC_RPATH first. So importing pyscf and dqclibs into one process cross-links
the two stacks according to import order.

Measured on macOS 15 (arm64), Python 3.14, pyscf 2.13.1:

import pyscf, dqclibs; dqclibs.CPBC()
# OSError: dlopen(.../dqclibs/libpbc.dylib): Symbol not found:
#   _GTO_aopair_lazy_contract   (Expected in: .../pyscf/lib/libcgto.dylib)

dqclibs' own libcgto exports that symbol; pyscf's does not.

import dqclibs; dqclibs.CGTO(); import pyscf; dqclibs.CPBC()
# loads, then SIGSEGV in PBC_ft_latsum_drv -- and before it crashes,
# periodic integrals silently return WRONG NUMBERS

The silent-wrong-numbers mode is the reason this is worth fixing rather than
documenting: in the downstream DQC test suite, a periodic KS energy changed
merely because an earlier test in the same process had imported pyscf.

pyscf itself is immune precisely because it links its own siblings by
@loader_path.

Fix

Two hunks, both inside if (APPLE) — no effect on Linux or Windows.

  1. libs/CMakeLists.txtset(CMAKE_INSTALL_NAME_DIR "@loader_path").
    Covers libcgto, libcvhf, libpbc, libnp_helper, libsymm.

  2. libs/libcint/CMakeLists.txt — a POST_BUILD install_name_tool -id.
    libcint needs a different form: it is built through ExternalProject_Add
    straight into CMAKE_LIBRARY_OUTPUT_DIRECTORY with no install() step, so
    CMAKE_INSTALL_NAME_DIR never applies to it. Setting MACOSX_RPATH OFF
    instead (tried both as a -D from the parent and as a target property)
    makes CMake fall back to an absolute build-tree path, which is worse
    than the @rpath it replaces. Rewriting the id after the link is the only
    form that survives both.

    Worth noting separately: libcint.dylib, libcint.4.dylib and
    libcint.4.0.7.dylib all carry the single id @rpath/libcint.4.dylib, so
    they collide with each other as well as with pyscf.

Verification

From a clean build:

  • zero @rpath references across all eight dylibs, ids and dependencies
  • dqclibs.CPBC() loads in both import orders
  • downstream (DQC): 28 periodic tests go from skipped to passing, and a
    periodic KS energy no longer depends on whether an earlier test imported
    pyscf

Rebuilding needs rm -rf dqclibs/deps first. ExternalProject_Add
installs libcint into ${PROJECT_SOURCE_DIR}/deps — inside the source
tree, not the build directory — so a plain rebuild silently reuses a stale
libcint and CMake changes to it appear to do nothing. This cost me several
rebuilds and made an earlier diagnosis look inconsistent.

Note for review

The codesign -f -s - ... || true after the id rewrite is deliberately
non-fatal (macOS invalidates the signature when the load commands change), but
it is a no-op where codesign is absent — happy to guard it or drop it if you
would rather not shell out.

On macOS every library here is stamped with an @rpath/<name>.dylib install
name (CMAKE_MACOSX_RPATH ON), and pyscf ships libcgto, libnp_helper, libcvhf
and libpbc under exactly those names. dyld satisfies an @rpath dependency from
any already-loaded image whose install name matches -- it does not consult the
referring library's own LC_RPATH first -- so importing pyscf and dqclibs into
one process cross-links the two stacks according to import order.

Measured on macOS 15 (arm64), python 3.14, pyscf 2.13.1:

  import pyscf; import dqclibs; dqclibs.CPBC()
      OSError: dlopen(.../dqclibs/libpbc.dylib): Symbol not found:
      _GTO_aopair_lazy_contract   (Expected in: .../pyscf/lib/libcgto.dylib)
    dqclibs' own libcgto exports that symbol; pyscf's does not.

  import dqclibs; dqclibs.CGTO(); import pyscf; dqclibs.CPBC()
      loads, then SIGSEGV in PBC_ft_latsum_drv, and periodic integrals
      silently return WRONG NUMBERS rather than crashing.

  Either way, a periodic KS energy changed merely because an earlier
  calculation in the same process had imported pyscf.

pyscf is immune because it links its own siblings by @loader_path. This does
the same for dqclibs, in two parts:

  * libs/CMakeLists.txt -- CMAKE_INSTALL_NAME_DIR "@loader_path" covers
    libcgto, libcvhf, libpbc, libnp_helper and libsymm.

  * libs/libcint/CMakeLists.txt -- libcint needs a POST_BUILD
    install_name_tool -id instead. It is built through ExternalProject_Add
    straight into CMAKE_LIBRARY_OUTPUT_DIRECTORY with no install() step, so
    CMAKE_INSTALL_NAME_DIR never applies to it; and setting MACOSX_RPATH OFF
    (tried both as a -D from the parent and as a target property) makes cmake
    fall back to an absolute build-tree path, which is worse than the @rpath it
    replaces. Rewriting the id after the link is the only form that survives
    both. Note also that libcint.dylib, libcint.4.dylib and libcint.4.0.7.dylib
    all share the single id @rpath/libcint.4.dylib, so they collide with each
    other as well as with pyscf.

Verified from a clean build (rm -rf dqclibs/deps first -- ExternalProject_Add
installs libcint into ${PROJECT_SOURCE_DIR}/deps, inside the SOURCE tree, so a
plain rebuild silently reuses a stale libcint):

  * zero @rpath references across all eight dylibs, ids and dependencies
  * dqclibs.CPBC() loads in both import orders
  * the downstream DQC test suite goes from 28 periodic tests skipped to 28
    passing, and a periodic KS energy no longer depends on whether an earlier
    test imported pyscf

No effect on Linux or Windows: both hunks are inside `if (APPLE)`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant