Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
b88ff39
Added .github/workflows/gpu_ci_trigger.yml
max-models May 28, 2026
9f0c910
run the gitlab workflow on the same branch
max-models May 28, 2026
ec18a94
push from github->gitlab
max-models May 28, 2026
e23cef9
trigger CI
max-models May 28, 2026
1c9322f
push with ssh
max-models May 28, 2026
7d2bd4f
Updated .gitlab-ci.yml
max-models May 28, 2026
856ef94
use a venv on gitalb ci
max-models May 28, 2026
677d04c
Use gitlab-registry.mpcdf.mpg.de/mpcdf/ci-module-image/nvhpcsdk_26:2026
max-models May 28, 2026
15ecddb
load modules
max-models May 28, 2026
8a55a18
cleanup
max-models May 28, 2026
ba45341
cleanup
max-models May 28, 2026
32a2362
Skip parity test
max-models May 28, 2026
a08068d
Wait for gitlab job completion
max-models May 28, 2026
e2557db
Split tests
max-models May 28, 2026
f190154
Fix gitlab ci trigger
max-models May 28, 2026
6be17e8
Sleep for 10s
max-models May 28, 2026
0210104
Simplify workflow
max-models May 28, 2026
dbfa33a
specify backend in tests
max-models May 28, 2026
89769ee
formatting
max-models May 28, 2026
7867891
Merge pull request #16 from max-models/11-add-a-one-way-mirror-to-mpc…
max-models May 28, 2026
8d65bac
Added some instructions to the ci trigger
max-models May 28, 2026
241a648
Added more tests
max-models May 28, 2026
6e48cb7
Fixed tests
max-models May 28, 2026
ca6746f
formatting
max-models May 28, 2026
c730709
fix tests
max-models May 28, 2026
f5ecc7a
load fft
max-models May 28, 2026
a59f36c
temp: added has cupy cache
max-models May 28, 2026
834bbc7
More robust cupy import
max-models May 28, 2026
903467c
Added has_cupy() method
max-models May 28, 2026
851a0b2
Run pytest -xvs
max-models May 28, 2026
ec04a25
Renamed has_cupy to cupy_available
max-models May 28, 2026
64ffb33
skip matmul test to check if cupy is available
max-models May 28, 2026
97e3b1e
Install nvidia-cublas-cu12
max-models May 28, 2026
54f5baa
Remove the matmul test to check for _CUPY_AVAILABLE_CACHE
max-models May 28, 2026
30350d0
Run gitlab ci using the CLI
max-models May 28, 2026
0572cc4
Replaced glab --wait with --live
max-models May 28, 2026
e78cef7
Added gpu-test prefix
max-models May 28, 2026
ac468e5
Merge pull request #17 from max-models/more-gpu-tests
max-models May 28, 2026
c4536d5
Set self._backend to cupy correctly and improved check_backend
max-models Jul 31, 2026
383ef87
Added __version__
max-models Jul 31, 2026
3a6e908
Added tests/unit/test_cunumpy.py
max-models Jul 31, 2026
daef0a5
Removed unneccesary post_init
max-models Jul 31, 2026
1c052ec
assert --> Raise, added docstring
max-models Jul 31, 2026
365cc65
Updated version number
max-models Jul 31, 2026
276f081
Merge remote-tracking branch 'origin/devel' into backend-fixes
max-models Jul 31, 2026
c137c41
Updated CI and changelog + small fixes
max-models Jul 31, 2026
aa09be3
Formatting
max-models Jul 31, 2026
b61fc82
ruff check fixes
max-models Jul 31, 2026
7bfe1ea
Fix cupy test, backend remained from old test
max-models Jul 31, 2026
9cefaea
Merge pull request #18 from max-models/backend-fixes
max-models Jul 31, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
81 changes: 81 additions & 0 deletions .github/workflows/gpu_ci_trigger.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
# SETUP INSTRUCTIONS:
# ------------------
# This workflow synchronizes the code to GitLab via SSH to trigger GPU-enabled CI.
#
# 1. GENERATE SSH KEY PAIR (on your local machine):
# ssh-keygen -t ed25519 -f ~/.ssh/gitlab_sync_key -N "" -C "github-to-gitlab-sync"
#
# 2. CONFIGURE GITLAB (The Target):
# - Go to GitLab project > Settings > Repository > Deploy keys.
# - Add the content of '~/.ssh/gitlab_sync_key.pub'.
# - IMPORTANT: Check "Allow write access to this repository".
#
# 3. CONFIGURE GITHUB (The Source):
# - Go to GitHub repo > Settings > Secrets and variables > Actions.
# - Add new Repository Secrets:
# - Name: GITLAB_SSH_PRIVATE_KEY
# Value: Paste the entire content of '~/.ssh/gitlab_sync_key'.
# - Name: GITLAB_TOKEN
# Value: Your GitLab Personal Access Token (with 'api' and 'read_repository' scopes).
#

name: Sync to GitLab and Run GPU CI

on:
push:
branches: [main, devel]
pull_request:
branches: [main, devel]
workflow_dispatch:

jobs:
sync-and-test:
runs-on: ubuntu-latest
steps:
- name: Checkout Code
uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Install SSH Key
uses: webfactory/ssh-agent@v0.9.0
with:
ssh-private-key: ${{ secrets.GITLAB_SSH_PRIVATE_KEY }}

- name: Push to GitLab via SSH & Provide Link
id: push
run: |
# 1. Setup SSH known hosts
mkdir -p ~/.ssh
ssh-keyscan gitlab.mpcdf.mpg.de >> ~/.ssh/known_hosts

# 2. Determine target branch
if [ "${{ github.event_name }}" == "pull_request" ]; then
TARGET_BRANCH="gpu-test-pr-${{ github.event.number }}"
else
SOURCE_REF="${{ github.ref_name }}"
SAFE_REF="${SOURCE_REF//\//-}"
TARGET_BRANCH="gpu-test-${SAFE_REF}"
fi
echo "TARGET_BRANCH=$TARGET_BRANCH" >> $GITHUB_ENV

# 3. Add GitLab SSH remote
git remote add gitlab git@gitlab.mpcdf.mpg.de:maxlin/cunumpy.git

# 4. Force push (This automatically starts the GitLab Pipeline)
git push -f gitlab HEAD:refs/heads/$TARGET_BRANCH

# 5. Provide the direct link
PIPELINE_URL="https://gitlab.mpcdf.mpg.de/maxlin/cunumpy/-/pipelines?ref=$TARGET_BRANCH"

echo "::notice::GitLab GPU CI Pipeline started automatically via Push!"
echo "::notice::View Pipeline: $PIPELINE_URL"

- name: Wait for GitLab Pipeline
uses: docker://gitlab/glab:latest
env:
GITLAB_TOKEN: ${{ secrets.GITLAB_TOKEN }}
GITLAB_HOST: gitlab.mpcdf.mpg.de
with:
entrypoint: glab
args: ci status --live --branch ${{ env.TARGET_BRANCH }} --repo maxlin/cunumpy
3 changes: 1 addition & 2 deletions .github/workflows/static_analysis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -78,15 +78,14 @@ jobs:

ruff:
runs-on: ubuntu-latest
continue-on-error: true
steps:
- name: Checkout the code
uses: actions/checkout@v4

- name: Linting with ruff
run: |
pip install ruff
ruff check src/ || true
ruff check src/

pylint:
runs-on: ubuntu-latest
Expand Down
11 changes: 8 additions & 3 deletions .github/workflows/testing.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,11 @@ jobs:
build:
runs-on: ubuntu-latest

strategy:
fail-fast: false
matrix:
python-version: ["3.8", "3.10", "3.13"]

steps:
# Checkout the repository
- name: Checkout code
Expand All @@ -23,16 +28,16 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.10' # Adjust as needed
python-version: ${{ matrix.python-version }}

# Cache pip dependencies
- name: Cache pip
uses: actions/cache@v3
with:
path: ~/.cache/pip
key: ${{ runner.os }}-pip-${{ hashFiles('**/pyproject.toml') }}
key: ${{ runner.os }}-pip-${{ matrix.python-version }}-${{ hashFiles('**/pyproject.toml') }}
restore-keys: |
${{ runner.os }}-pip-
${{ runner.os }}-pip-${{ matrix.python-version }}-

- name: Install dependencies
run: |
Expand Down
99 changes: 44 additions & 55 deletions .gitlab-ci.yml
Original file line number Diff line number Diff line change
@@ -1,57 +1,46 @@
# This file is a template, and might need editing before it works on your project.
# To contribute improvements to CI/CD templates, please follow the Development guide at:
# https://docs.gitlab.com/ee/development/cicd/templates.html
# This specific template is located at:
# https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Python.gitlab-ci.yml

# Official language image. Look for the different tagged releases at:
# https://hub.docker.com/r/library/python/tags/
image: python:latest

# Change pip's cache directory to be inside the project directory since we can
# only cache local items.
variables:
PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pip"

# https://pip.pypa.io/en/stable/topics/caching/
cache:
paths:
- .cache/pip

before_script:
- python --version ; pip --version # For debugging
- pip install virtualenv
- virtualenv venv
- source venv/bin/activate

test:
# Use the specialized MPCDF HPC image
CUDA_IMAGE: "gitlab-registry.mpcdf.mpg.de/mpcdf/ci-module-image/nvhpcsdk_26:2026"
PIP_DISABLE_PIP_VERSION_CHECK: "1"

stages:
- test

gpu_tests:
stage: test
image: ${CUDA_IMAGE}
tags:
- gpu-nvidia
- gpu-nvidia-cc80
before_script:
- module load python-waterboa/2025.06
- module load nvhpcsdk/26
- module load fftw-serial/3.3.10
script:
- pip install ruff tox # you can also use tox
- pip install --editable ".[test]"
- tox -e py,ruff

run:
script:
- pip install .
# run the command here
artifacts:
paths:
- build/*

pages:
script:
- pip install sphinx sphinx-rtd-theme
- cd doc
- make html
- mv build/html/ ../public/
artifacts:
paths:
- public
rules:
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

deploy:
stage: deploy
script: echo "Define your deployment script!"
environment: production

- echo "--- CUDA Sanity Check ---"
- nvidia-smi

- echo "--- Detect Compute Capability ---"
- nvidia-smi --query-gpu=compute_cap --format=csv,noheader

- echo "--- Tool Versions ---"
- cmake --version
- python3 --version
- git --version

- echo "--- Pytest Execution ---"
# The MPCDF image likely has a specific python environment.
# We install our dependencies into the user directory or a virtualenv.
- python3 -m pip install --user cupy-cuda12x
- python3 -m pip install --user nvidia-cublas-cu12 nvidia-cufft-cu12 nvidia-curand-cu12 nvidia-cusolver-cu12 nvidia-cusparse-cu12
- python3 -m pip install --user -e .

# Add the user bin to PATH for pytest
- export PATH="$HOME/.local/bin:$PATH"

# Try to find libcublas and other libraries in the HPC environment
- export LD_LIBRARY_PATH=$(find /mpcdf/soft /opt/nvidia -name libcublas.so.12 -exec dirname {} \; 2>/dev/null | head -n 1):$LD_LIBRARY_PATH

- export ARRAY_BACKEND=cupy

- pytest -xvs .
17 changes: 17 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,22 @@ All notable changes to the `cunumpy` library are documented here.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [0.1.3] - 2026-07-31

### Added
- `xp.set_device(device_id)`: Select the active CUDA device for the current process (no-op on the NumPy backend).
- `xp.__version__`: Reports the installed `cunumpy` package version.

### Fixed
- `ArrayBackend` no longer silently reports `"cupy"` as the active backend when CuPy was requested but is unavailable; it now correctly falls back to reporting `"numpy"` so `xp.cupy_backend`/`xp.numpy_backend` reflect what actually loaded.
- `to_numpy()` no longer misdetects CPU objects that merely expose a `.get` method (e.g. dict-like objects) as CuPy arrays; it now checks `get_backend()` instead of `hasattr(array, "get")`.
- Invalid backend names now raise `ValueError` instead of relying on a bare `assert`, which was previously stripped under `python -O`.

### Changed
- Removed the redundant `ArrayBackend.__init_post__` double-initialization path.
- `ArrayBackend` documents that it is not thread-safe (global mutable backend state).
- CI now runs the test suite across a Python 3.8/3.10/3.13 matrix instead of only 3.10, and `ruff` is now an enforced check rather than advisory.

## [0.1.2] - 2026-05-27

### Added
Expand All @@ -22,6 +38,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- `xp.synchronize()`: Blocks until GPU operations are complete (no-op on CPU). Essential for accurate benchmarking.
- **Developer Experience**:
- Added `isort` configuration to `pyproject.toml` with `black` profile compatibility.
- Reorganized test suite into specialized files: `test_numpy.py`, `test_cupy.py`, and `test_cunumpy.py`.

### Changed
- **Dynamic Dispatch Architecture**: Refactored `src/cunumpy/xp.py` to use module-level `__getattr__`. This ensures that `cunumpy.<op>` calls always resolve to the currently active backend module, enabling seamless runtime switching via `set_backend`.
Expand Down
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,8 @@ export ARRAY_BACKEND=cupy

```python
import cunumpy as xp
arr = xp.array([1,2])

arr = xp.array([1, 2])

print(type(arr))
print(xp.__version__)
Expand Down
2 changes: 1 addition & 1 deletion docs/source/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ xp.set_backend("cupy")
# Scoped backend switching
with xp.use_backend("numpy"):
arr_cpu = xp.zeros(10)
print(xp.is_cpu(arr_cpu)) # True
print(xp.is_cpu(arr_cpu)) # True

# Explicit conversion
arr_np = xp.to_numpy(arr)
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ requires = [ "setuptools", "wheel" ]

[project]
name = "cunumpy"
version = "0.1.2"
version = "0.1.3"
description = "Simple wrapper for numpy and cupy. Replace `import numpy as np` with `import cunumpy as xp`."
readme = "README.md"
keywords = [ "python" ]
Expand Down
28 changes: 20 additions & 8 deletions src/cunumpy/__init__.py
Original file line number Diff line number Diff line change
@@ -1,30 +1,42 @@
# cunumpy/__init__.py
from importlib.metadata import PackageNotFoundError, version

from . import xp
from .xp import (
cupy_available,
get_backend,
is_cpu,
is_gpu,
set_backend,
set_device,
synchronize,
to_cunumpy,
to_cupy,
to_numpy,
use_backend,
)

try:
__version__ = version("cunumpy")
except PackageNotFoundError:
__version__ = "0.0.0+unknown"

__all__ = [
"xp",
"to_numpy",
"to_cupy",
"to_cunumpy",
"__version__",
"cupy_available",
"cupy_backend",
"get_backend",
"is_gpu",
"is_cpu",
"use_backend",
"is_gpu",
"numpy_backend",
"set_backend",
"set_device",
"synchronize",
"numpy_backend",
"cupy_backend",
"to_cunumpy",
"to_cupy",
"to_numpy",
"use_backend",
"xp",
]


Expand Down
7 changes: 5 additions & 2 deletions src/cunumpy/__init__.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -7,18 +7,21 @@ from typing import Any, Generator
import numpy as np
from numpy import *

from . import xp
from . import xp as xp

def to_numpy(array: Any) -> np.ndarray: ...
def to_cupy(array: Any) -> Any: ...
def to_cunumpy(array: Any) -> Any: ...
def cupy_available() -> bool: ...
def get_backend(array: Any) -> str: ...
def is_gpu(array: Any) -> bool: ...
def is_cpu(array: Any) -> bool: ...
@contextmanager
def use_backend(backend: str) -> Generator[None, None, None]: ...
def use_backend(backend: str) -> Generator[None]: ...
def set_backend(backend: str) -> None: ...
def set_device(device_id: int) -> None: ...
def synchronize() -> None: ...

numpy_backend: bool
cupy_backend: bool
__version__: str
Loading
Loading