-
Notifications
You must be signed in to change notification settings - Fork 0
Add QUIC operator transport #9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
danprudky
wants to merge
4
commits into
main
Choose a base branch
from
BAF-1744/GUI-RTSP-server-streaming-control
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
b731725
feat: add QUIC operator transport support (BAF-1744)
dff087c
fix(BAF-1744): harden QUIC operator status/command handling
f4b40e6
feat(BAF-1744): switch QUIC operator wire format to protobuf
507fa9f
fix(BAF-1744): opt module manager into MG push-only forwarding
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
60 changes: 60 additions & 0 deletions
60
include/bringauto/transparent_module_utils/operator_stream/OperatorChannel.hpp
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,60 @@ | ||
| #pragma once | ||
|
|
||
| #include <chrono> | ||
| #include <condition_variable> | ||
| #include <cstdint> | ||
| #include <mutex> | ||
| #include <optional> | ||
| #include <string> | ||
| #include <vector> | ||
|
|
||
| namespace bringauto::transparent_module_utils::operator_stream { | ||
|
|
||
| /** | ||
| * One command received from an operator over QUIC, handed across the thread boundary to the | ||
| * synchronous ES API. `payload` is the opaque command buffer (today: the BAF-1651 JSON envelope) | ||
| * passed through verbatim; device_* are the fleet-protocol device coordinates kept for routing. | ||
| * | ||
| * Ported from teleop-module's OperatorChannel (BAF-1670) for the Transparent module's QUIC | ||
| * operator transport (BAF-1744) — see that repo's equivalent file for the original design notes. | ||
| */ | ||
| struct OperatorCommand { | ||
| std::vector<std::uint8_t> payload; | ||
| std::int64_t timestamp_ms{0}; | ||
| std::uint32_t module_id{0}; | ||
| std::uint32_t device_type{0}; | ||
| std::string device_role; | ||
| std::string device_name; | ||
| }; | ||
|
|
||
| /** | ||
| * Thread-safe seam between the QUIC server's ASYNCHRONOUS msquic receive callback (producer) and | ||
| * the module's SYNCHRONOUS, blocking external-server API (consumer) — see | ||
| * QuicOperatorServer for the producer side and external_server_api.cpp's wait_for_command() for | ||
| * the consumer side. | ||
| */ | ||
| class OperatorChannel { | ||
| public: | ||
| /// Producer (QUIC recv-callback thread): hand over a command and wake a waiter. Coalesces to | ||
| /// the newest — a pending unconsumed command is dropped (freshest wins). No-op after shutdown(). | ||
| void push(OperatorCommand command); | ||
|
|
||
| /// Consumer (ES `wait_for_command` thread): block up to `timeout` for the latest command. | ||
| /// Returns std::nullopt on timeout or after shutdown(). | ||
| [[nodiscard]] std::optional<OperatorCommand> waitForCommand(std::chrono::milliseconds timeout); | ||
|
|
||
| /// Drop queued commands — e.g. on operator disconnect. | ||
| void clear(); | ||
|
|
||
| /// Unblock all waiters and refuse further waits/pushes (module teardown). | ||
| void shutdown(); | ||
|
|
||
| private: | ||
| std::mutex mutex_; | ||
| std::condition_variable cv_; | ||
| /// Single coalescing slot: push() always supersedes any unconsumed command, so at most one is held. | ||
| std::optional<OperatorCommand> pending_; | ||
| bool shutdown_{false}; | ||
| }; | ||
|
|
||
| } // namespace bringauto::transparent_module_utils::operator_stream |
132 changes: 132 additions & 0 deletions
132
include/bringauto/transparent_module_utils/operator_stream/QuicOperatorServer.hpp
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,132 @@ | ||
| #pragma once | ||
|
|
||
| #include <bringauto/transparent_module_utils/operator_stream/OperatorChannel.hpp> | ||
|
|
||
| #include <msquic.h> | ||
|
|
||
| #include <atomic> | ||
| #include <cstdint> | ||
| #include <mutex> | ||
| #include <string> | ||
|
|
||
| namespace bringauto::transparent_module_utils::operator_stream { | ||
|
|
||
| /** | ||
| * Configuration for the operator-facing QUIC server. mTLS optional (disable client auth + | ||
| * self-signed server cert for loopback bring-up). | ||
| */ | ||
| struct QuicOperatorServerConfig { | ||
| std::uint16_t port{0}; | ||
| std::string alpn{"transparent-operator"}; | ||
| std::string certPath; // server certificate (PEM) | ||
| std::string keyPath; // server private key (PEM) | ||
| std::string caCertsPath; // CA for verifying operator client certs (empty = no CA file) | ||
| bool disableClientAuth{false}; // true for loopback/dev; false requires operator client certs | ||
| std::string listenAddress; // empty = any | ||
| }; | ||
|
|
||
| /** | ||
| * Operator-facing QUIC server living inside the Transparent module's external-server plugin — | ||
| * the operator leg, used alongside (not instead of) the existing Fleet HTTP API path (BAF-1744; | ||
| * ported from teleop-module's QuicOperatorServer, BAF-1670, single-operator Phase 1). | ||
| * | ||
| * Wire format is protobuf (proto/transparent_operator_stream.proto's OperatorMessage), matching | ||
| * teleop-module's QuicOperatorServer this was ported from — but its own .proto/package, since | ||
| * external-server-cpp dlopens both teleop-external-server-shared.so and | ||
| * transparent-external-server-shared.so into the same process, and protobuf's generated-descriptor | ||
| * registry is process-global: two .so's registering the same .proto file/package would crash. The | ||
| * BAF-1651 JSON envelope itself still rides as opaque `bytes` inside OperatorMessage's | ||
| * StatusUpdate/CommandRequest — this class stays decoupled from that schema. | ||
| * | ||
| * cloud -> operator : sendStatus() opens a short unidirectional stream per OperatorMessage{status} | ||
| * (START|FIN), freeing the send buffer on SEND_COMPLETE. | ||
| * operator -> cloud : each operator command arrives as a unidirectional stream; the receive | ||
| * callback accumulates the bytes, parses OperatorMessage{command}, and hands | ||
| * it to the OperatorChannel (which unblocks the module's wait_for_command()). | ||
| * | ||
| * Threading: msquic invokes the callbacks on its own worker thread(s). Commands cross into the | ||
| * synchronous module API through the OperatorChannel (the thread-safe seam). sendStatus() is called | ||
| * on the ES thread (from forward_status) and msquic StreamSend is thread-safe. | ||
| */ | ||
| class QuicOperatorServer { | ||
| public: | ||
| QuicOperatorServer(QuicOperatorServerConfig config, OperatorChannel &channel); | ||
| ~QuicOperatorServer(); | ||
|
|
||
| QuicOperatorServer(const QuicOperatorServer &) = delete; | ||
| QuicOperatorServer &operator=(const QuicOperatorServer &) = delete; | ||
|
|
||
| /// Open msquic, registration, configuration, credential (mTLS), and the listener. False on any failure. | ||
| [[nodiscard]] bool initialize(); | ||
|
|
||
| /// Start the listener. False on failure. | ||
| [[nodiscard]] bool start(); | ||
|
|
||
| /// Stop the listener + drop the operator connection. | ||
| void stop(); | ||
|
|
||
| /// Outcome of sendStatus(). NoOperator is an expected best-effort drop (nobody to send to), | ||
| /// distinct from SendFailed (serialize / StreamOpen / StreamSend error) so the caller can react. | ||
| enum class SendResult { Sent, NoOperator, SendFailed }; | ||
|
|
||
| /// Send one status to the connected operator (best-effort: NoOperator if none). Called from the | ||
| /// ES thread. payload is the opaque status buffer (today: JSON); device_* are the fleet-protocol coords. | ||
| SendResult sendStatus(std::uint32_t module_id, std::uint32_t device_type, const std::string &device_role, | ||
| const std::string &device_name, const std::uint8_t *payload, std::size_t payload_size, | ||
| std::int64_t timestamp_ms); | ||
|
|
||
| [[nodiscard]] bool hasOperator() const { return operatorConnection_.load() != nullptr; } | ||
|
|
||
| private: | ||
| /// Heap buffer kept alive across an async StreamSend; freed in the send callback on SEND_COMPLETE. | ||
| struct SendBuffer { | ||
| explicit SendBuffer(std::string &&data) : payload(std::move(data)) { | ||
| buffer.Length = static_cast<std::uint32_t>(payload.size()); | ||
| buffer.Buffer = reinterpret_cast<std::uint8_t *>(payload.data()); | ||
| } | ||
| std::string payload; | ||
| QUIC_BUFFER buffer{}; | ||
| }; | ||
|
|
||
| /// Per-inbound-stream accumulation context (a command may arrive in multiple RECEIVE events). | ||
| struct InboundStreamContext { | ||
| QuicOperatorServer *server{nullptr}; | ||
| HQUIC connection{nullptr}; // owning connection; commands are accepted only while it is the operator | ||
| std::string bytes; | ||
| }; | ||
|
|
||
| bool loadQuic(); | ||
| bool initRegistration(); | ||
| bool initConfiguration(); | ||
| bool initCredential(); | ||
| bool initListener(); | ||
|
|
||
| /// Parse an accumulated command frame and hand it to the channel. | ||
| void handleCommandBytes(const std::string &bytes); | ||
|
|
||
| static QUIC_STATUS QUIC_API listenerCallback(HQUIC listener, void *context, QUIC_LISTENER_EVENT *event); | ||
| static QUIC_STATUS QUIC_API connectionCallback(HQUIC connection, void *context, QUIC_CONNECTION_EVENT *event); | ||
| static QUIC_STATUS QUIC_API commandStreamCallback(HQUIC stream, void *context, QUIC_STREAM_EVENT *event); | ||
| static QUIC_STATUS QUIC_API sendStreamCallback(HQUIC stream, void *context, QUIC_STREAM_EVENT *event); | ||
|
|
||
| QuicOperatorServerConfig config_; | ||
| OperatorChannel &channel_; | ||
|
|
||
| const QUIC_API_TABLE *quic_{nullptr}; | ||
| HQUIC registration_{nullptr}; | ||
| HQUIC quicConfig_{nullptr}; | ||
| HQUIC listener_{nullptr}; | ||
| QUIC_ADDR quicAddr_{}; | ||
| QUIC_BUFFER alpnBuffer_{}; | ||
|
|
||
| /// Single active operator connection (Phase 1: single operator). | ||
| std::atomic<HQUIC> operatorConnection_{nullptr}; | ||
| std::atomic<bool> running_{false}; | ||
|
|
||
| /// Serializes use of the connection handle (sendStatus' StreamOpen/StreamSend) against its | ||
| /// teardown (ConnectionClose in the SHUTDOWN_COMPLETE callback). The atomic above guards the | ||
| /// pointer value only, not the lifetime of the handle it points to. | ||
| std::mutex connectionMutex_; | ||
| }; | ||
|
|
||
| } // namespace bringauto::transparent_module_utils::operator_stream |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,58 @@ | ||
| syntax = "proto3"; | ||
|
|
||
| // GUI <-> transparent-module QUIC stream protocol (BAF-1744), modeled on teleop-module's | ||
| // operator_stream.proto (BAF-1670) — same message shapes, but its own package/file: external-server-cpp | ||
| // dlopens both teleop-external-server-shared.so and transparent-external-server-shared.so into the | ||
| // same process, and protobuf's generated-descriptor registry is process-global, so two .so's | ||
| // registering the same .proto file/package would crash. A distinct package avoids that collision. | ||
| // | ||
| // Transport: each message travels on its own UNIDIRECTIONAL QUIC stream between the GUI (client) | ||
| // and transparent-module's external-server plugin (server). The sender opens a stream, writes the | ||
| // serialized OperatorMessage, and closes it with FIN — the stream boundary frames the message, so | ||
| // there is NO length prefix and one stream carries exactly one OperatorMessage. | ||
| // | ||
| // Direction: | ||
| // rtsp-server -> GUI : StatusUpdate (device-reported status / BAF-1651 response envelope) | ||
| // GUI -> rtsp-server : CommandRequest (a BAF-1651 command envelope) | ||
| // | ||
| // The BAF-1651 JSON envelope ({type, version, action, id, timestamp, payload}) is passed through | ||
| // VERBATIM as opaque bytes — this protocol only carries the Fleet Protocol device_id + timestamp | ||
| // needed to route the message, and stays decoupled from the BAF-1651 schema. | ||
| // NB: `operator_stream`, not `operator` — `operator` is a C++ keyword (invalid generated namespace). | ||
| package bringauto.transparent.operator_stream; | ||
|
|
||
| // Fleet-protocol device coordinates of the addressed device (module 3 / type 1 / role rtsp_server). | ||
| message DeviceId { | ||
| uint32 module_id = 1; | ||
| uint32 type = 2; | ||
| string role = 3; | ||
| string name = 4; | ||
| } | ||
|
|
||
| // rtsp-server -> GUI: one status/response push. | ||
| message StatusUpdate { | ||
| DeviceId device = 1; | ||
| bytes payload = 2; // opaque BAF-1651 JSON envelope | ||
| int64 timestamp_ms = 3; | ||
| } | ||
|
|
||
| // GUI -> rtsp-server: one streaming-control command. | ||
| message CommandRequest { | ||
| DeviceId device = 1; | ||
| bytes payload = 2; // opaque BAF-1651 JSON envelope | ||
| int64 timestamp_ms = 3; | ||
| } | ||
|
|
||
| // GUI -> rtsp-server: sent once on connect. Single-operator (Phase 1 of the pattern this was | ||
| // ported from) — contents ignored for now. | ||
| message OperatorHello { | ||
| string operator_id = 1; | ||
| } | ||
|
|
||
| message OperatorMessage { | ||
| oneof kind { | ||
| OperatorHello hello = 1; // GUI -> rtsp-server | ||
| StatusUpdate status = 2; // rtsp-server -> GUI | ||
| CommandRequest command = 3; // GUI -> rtsp-server | ||
| } | ||
| } | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Buf lint failure: package path doesn't match file location.
Buf's
PACKAGE_DIRECTORY_MATCHrule requires the file for packagebringauto.transparent.operator_streamto live underbringauto/transparent/operator_stream/, but it sits inproto/. If buf lint gates CI this will fail the build.Options: relocate the file to
proto/bringauto/transparent/operator_stream/transparent_operator_stream.proto(and update thePROTOBUF_GENERATE_CPPpath inCMakeLists.txtline 81 plus the generated-header include), or exclude/ignore this rule inbuf.yamlif the flat layout is intentional.🧰 Tools
🪛 Buf (1.71.0)
[error] 22-22: Files with package "bringauto.transparent.operator_stream" must be within a directory "bringauto/transparent/operator_stream" relative to root but were in directory "proto".
(PACKAGE_DIRECTORY_MATCH)
🤖 Prompt for AI Agents
Source: Linters/SAST tools