cuda_buffer_backend::from_input_buffer should reject non-cuda backend buffer

[dskkato]

Description

Currently, cuda_buffer_backend::from_input_buffer accept non-cuda backend rosidl::Buffer, and when there is mismatch between backends, it automatically allocates CUDA memory, without touching the original buffer contents. Since the original buffer descriptor stays in the message field, the following scenario could happen:

• create a default CPU backend buffer
• feed it to cuda_buffer_backend::from_input_buffer
• use the CUDA memory through WriteHandle
• publish
  • For publish step, only the original buffer descriptor can be seen

There is no automatic descriptor conversion in the publish path, so I'd like to recommend to reject such a backend mismatch for the write side. For read side, the automatic conversion is still necessary since some tools, such as rosbag, will publish the buffer with the default cpu fallbacked backend.

Motivation

I'm currently working on shared memory based rosidl::Buffer backend, and thinking about combinations of various backends.

https://github.com/dskkato/memfd_buffer_backend

For now, I'm trying to follow this cuda_buffer_backend design, and realized a bunch of buffer backend combinations to those APIs.

Design / Implementation Considerations

I believe it is better to throw an exception for an unmatched rosidl::Buffer object.

Additional Information

No response

[dskkato]

Oh, sorry. I just realized that this behavior is explicitly described in the README.

That said, I still think this API can be error-prone, especially on the write path, so rejecting a backend mismatch may be safer.

However, if this auto-promotion behavior is intentional by design, please feel free to close this issue.

### Auto-promoting non-CUDA buffers

`from_input_buffer` / `from_output_buffer` accept any `rosidl::Buffer<T>`, not
just CUDA-backed ones. If the source is a non-CUDA buffer (e.g. the CPU
fallback path), a new CUDA-backed `rosidl::Buffer<uint8_t>` is allocated on the
fly and the returned handle points at it. The handle owns the promoted buffer;
call `handle.get_promoted_buffer()` to retrieve it.

[nvcyc]

Thanks for raising this question!
It's also very interesting to check out your memfd_buffer_backend and I'm looking forward to more discussion with you and the community on that topic.

The idea of allowing from_input_buffer/from_output_buffer in the CUDA buffer backend to accept non-CUDA-backed buffer is to increase the interoperability while keeping the user code simple, so the same node can be used in different pipelines and still work without needing to modify the source code.

I agree with you that it is error-prone by the case pointed out in your example.
Considering the benefit, I think this is a tolerable trade-off we'll embrace for now, but certainly we can do better to make this limitation clearer in the doc.

For allocating a new message, I'd like to point out that there is a "conversion" package layer in this software stack where we think most node developers would be using at the message and library level without needing to worry too much about the "backend" level issue.

For example, for LibTorch library users, the torch_conversions package offers APIs like allocate_tensor_msg() to create a message of a supported type (tensor_msgs/ExperimentalTensor in this case) that contains a buffer backed by CUDA buffer backend and is compatible with LibTorch through the provided APIs. Another good example is this PR #17 that adds support for sensor_msgs/Image.

Since most buffers are only meaningful when interpreted together with the accompany fields in the same message, it makes more sense for users to stay at the message and library level with this type of conversion packages/APIs.

It's also worth noting that the APIs like allocate_tensor_msg() are technically backend-agnostic from the interface perspective, so the node level user code can actually be backend-agnostic and focus on the message and library level handling.