Avoid same-process CUDA publish synchronization - #18
Open
nvcyc wants to merge 2 commits into
Open
Conversation
nvcyc
marked this pull request as ready for review
August 28, 2026 16:16
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR removes an unnecessary publish-path synchronization when CUDA-backed messages are transported between nodes in the same process through the regular DDS path.
Previously, the CUDA serialization backend attempted to use a CUDA IPC event for every descriptor. CUDA IPC event handles cannot be reopened in the process that created them, so same-process descriptor
transport called
cudaEventSynchronize()before publishing. This preserved correctness but blocked the publisher until all queued CUDA work completed.Same-process event transport
The CUDA allocation pool now owns a local CUDA event for each IPC-capable VMM block.
When serializing a descriptor for a same-process endpoint, the backend:
cudaEvent_tin the descriptor.The receiving backend attaches this event to the imported CUDA buffer as a non-owning write event. A subsequent
ReadHandlewaits for it on the consumer stream, preserving stream ordering withoutblocking the publisher thread.
If a local event cannot be created, the backend retains the previous
cudaEventSynchronize()behavior as a correctness-preserving fallback.Descriptor event types
CudaBufferDescriptornow explicitly identifies how its event data should be interpreted:EVENT_NONE— no producer event is attached.EVENT_LOCAL— the descriptor contains a same-processcudaEvent_t.EVENT_IPC— the descriptor contains an inter-processcudaIpcEventHandle_t.This avoids relying on PID comparisons, which can be incorrect across container PID namespaces.
Inter-process transport continues to use CUDA IPC events. Imported IPC events remain receiver-owned, while same-process local events remain owned by the publisher’s allocation pool.
Regression testing
This PR adds a dedicated Fast RTPS launch test for same-process descriptor transport.
The test:
publish()returns before the queued CUDA work completes.ReadHandle.Before the fix, the regression test receives valid CUDA payloads but fails because every publication reports
nonblocking=false.After the fix, all five publications report
nonblocking=true, and all received CUDA payloads validate successfully.