Skip to content

Add cuda_buffer_py - #6

Open
nvcyc wants to merge 2 commits into
mainfrom
nvcyc/cuda_buffer_py
Open

Add cuda_buffer_py#6
nvcyc wants to merge 2 commits into
mainfrom
nvcyc/cuda_buffer_py

Conversation

@nvcyc

@nvcyc nvcyc commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Description

This PR adds Python bindings for the CUDA buffer backend, enabling rclpy publishers and subscribers to exchange CUDA-backed ROS message fields without converting them to CPU memory.

The new cuda_buffer_py package exposes CUDA allocation and scoped read/write APIs that follow the existing C++ buffer lifecycle model.

Python API

The package adds:

  • CudaBuffer.from_cpu(data) — copy CPU data into a CUDA-backed buffer.
  • CudaBuffer.from_size(size) — create a zero-initialized CUDA-backed buffer.
  • CudaBuffer.allocate_buffer(size) — allocate uninitialized CUDA storage without synchronization.
  • CudaBuffer.from_output_buffer(buffer, stream) — acquire a CudaWriteHandle.
  • CudaBuffer.from_input_buffer(buffer, stream) — acquire a CudaReadHandle.

CudaReadHandle and CudaWriteHandle are context managers that expose the CUDA device pointer through device_ptr and get_ptr().

Python type stubs and a py.typed marker are included.

Synchronization and lifecycle

The scoped Python handles preserve the synchronization and lifetime guarantees of the C++ implementation:

  • Closing a write handle records producer completion on the supplied CUDA stream.
  • Acquiring a read handle waits for the producer event on the consumer stream.
  • Closing a read handle records consumer completion before the allocation can be recycled.
  • Handles retain ownership of the underlying or promoted buffer for the required lifetime.
  • CUDA-backed input remains on the GPU without CPU conversion.
  • CUDA-to-CUDA synchronization uses stream-ordered CUDA events without synchronizing the producer or consumer stream.

CPU-backed inputs can still be promoted to CUDA automatically. The from_cpu() and from_size() convenience APIs synchronize initialization before returning, while allocate_buffer() and the scoped-
handle path support asynchronous device-only workflows.

The bindings use the process-wide CUDA allocation pool provided by the installed cuda_buffer shared library. This allows the Python extension and dynamically loaded serialization backend to resolve
the same CUDA allocations during descriptor creation.

rclpy usage

Publishers can allocate and populate message fields directly on the GPU:

msg.data = CudaBuffer.allocate_buffer(size)

with CudaBuffer.from_output_buffer(msg.data, stream_ptr) as handle:
    produce_device_data(handle.device_ptr, size, stream_ptr)

publisher.publish(msg)

Subscribers opt into the CUDA backend and consume the device pointer directly:

subscription = node.create_subscription(
    Image,
    'image',
    callback,
    10,
    acceptable_buffer_backends='cuda',
)

def callback(msg):
    with CudaBuffer.from_input_buffer(msg.data, stream_ptr) as handle:
        consume_device_data(handle.device_ptr, len(msg.data), stream_ptr)

Testing

The package includes unit tests for allocation, conversion, scoped handles, synchronization, error handling, and lifetime management.

It also includes a Fast RTPS multiprocess regression test that:

  • Runs the Python publisher and subscriber in separate OS processes.
  • Uses regular DDS inter-process transport.
  • Publishes five CUDA-backed messages with known payloads.
  • Checks that each received field reports the cuda backend before creating a read handle.
  • Copies the final data to host and validates every payload.
  • Fails if the backend falls back to CPU, payload validation fails, or fewer than five explicit validations are observed.

@nvcyc
nvcyc marked this pull request as ready for review August 10, 2026 17:54
@nvcyc
nvcyc force-pushed the nvcyc/cuda_buffer_py branch 4 times, most recently from 4aae907 to c18390d Compare August 19, 2026 07:51
Comment thread cuda_buffer_backend/cuda_buffer_py/src/cuda_buffer_py.cpp Outdated
@nvcyc
nvcyc force-pushed the nvcyc/cuda_buffer_py branch from c18390d to 1c05282 Compare August 27, 2026 18:01
Signed-off-by: CY Chen <cyc@nvidia.com>
@nvcyc
nvcyc force-pushed the nvcyc/cuda_buffer_py branch from 1c05282 to 3ff611f Compare August 27, 2026 21:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants