FIX: Use subprocess isolation for macOS torch tests instead of skipping#4430
Closed
Heyy-Himanshuu wants to merge 1 commit into
Closed
FIX: Use subprocess isolation for macOS torch tests instead of skipping#4430Heyy-Himanshuu wants to merge 1 commit into
Heyy-Himanshuu wants to merge 1 commit into
Conversation
On macOS, torch tests segfault due to an OpenMP/libomp conflict between PyTorch and LightGBM (upstream: pytorch/pytorch#121101). Previously, all 17 affected tests were skipped entirely on Darwin, resulting in zero torch test coverage on macOS CI. This commit adds subprocess isolation to conftest.py that: - Detects skipif markers referencing GH #4075 at collection time - Replaces them with a subprocess_isolation marker on macOS - Runs those tests in a fresh subprocess (clean OpenMP state) - Reports segfaults/timeouts as FAILED instead of hanging the suite - Uses an environment variable to prevent recursion in the child No test files are modified — the hook intercepts markers dynamically. Closes #4075
CloseChoice
requested changes
Apr 22, 2026
CloseChoice
left a comment
Collaborator
There was a problem hiding this comment.
Thanks for the PR and would be great if we could get the tests back to work, but I would prefer a decorator for the affected tests, something like:
@subprocess_isolated(reason="torch/OpenMP segfault on macOS, GH #4075")
The pytest magic is overly complex and error-prone.
Contributor
@CloseChoice Hi, I believe the failing macOS tests have been fixed by #4545. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fixes #4075
On macOS, 17 torch-related tests across 5 files are completely skipped due to a known upstream PyTorch/OpenMP segfault (pytorch/pytorch#121101). When run without skipping, the tests either segfault or hang indefinitely, killing the entire CI suite.
This PR adds subprocess isolation to
tests/conftest.pythat:skipifmarkers referencing GH TRACKER: MacOS segmentation fault for torch test #4075 at collection time viapytest_collection_modifyitemssubprocess_isolationmarkersubprocess.run) with a 5-minute timeoutFAILEDwith diagnostics instead of hanging the suite_SHAP_SUBPROCESS_CHILD) to prevent recursion in the child processNo test files are modified — the hook intercepts markers dynamically at runtime. On Linux/Windows, the hooks are no-ops and tests run normally as before.
Why not
pytest-forked?pytest-forkedusesos.fork(), which is unsafe with multithreaded C libraries on macOS and can make the problem worse.How it works
_SHAP_SUBPROCESS_CHILDunset → replacesskipifwithsubprocess_isolation_SHAP_SUBPROCESS_CHILD=1→ removesskipifso test runs in-processsubprocess.run(...)subprocess_isolationmarkerChecklist