diff --git a/samples/c_samples/README.md b/samples/c_samples/README.md index f570573e8..7710016e8 100644 --- a/samples/c_samples/README.md +++ b/samples/c_samples/README.md @@ -17,3 +17,190 @@ make # On a live camera ./cannyDetect --live ``` + +## Graph Pipelining + +Demonstrates the OpenVX `vx_khr_pipelining` extension on a mixed CPU+GPU +workload. The same graph is executed synchronously (`--pipeline 0`) and with +`QUEUE_AUTO` pipelining (`--pipeline 1`) so the speed-up is visible. + +```shell +cd c_samples/pipelining/ +cmake . +make + +# Synchronous (baseline) — 4K heavy default workload +./pipelining --pipeline 0 + +# Pipelined — 4K heavy default workload +./pipelining --pipeline 1 + +# Compare both modes in one table +./pipelining --compare + +# Tune pipeline depth +./pipelining --pipeline 1 --pipeline-depth 2 + +# Lower resolution for faster runs +./pipelining --compare --mode light --resolution fhd --frames 60 +``` + +### What the sample does + +The graph processes a synthetic RGB video stream frame by frame. Two presets +are provided: + +* **Heavy (default)** — sized so the GPU has enough work to outperform the CPU: + ``` + RGB input -> ColorConvert -> IYUV -> ChannelExtract(Y) -> Gaussian3x3 -> U8 output + ``` +* **Light** — one filter pass for fast runs and bit-exact verification: + ``` + RGB input -> ColorConvert -> IYUV -> ChannelExtract(Y) -> Box3x3 -> U8 output + ``` + +`ColorConvert` and `ChannelExtract` are lightweight color-space operations, +while `Box3x3` and `Gaussian3x3` are heavier 2D convolutions. MIVisionX +automatically schedules nodes on the best available target (CPU or GPU), so +this graph naturally mixes host-side work with device-side work. + +Resolution presets are selectable with `--resolution hd|fhd|qhd|4k`: + +* `hd` = 1280×720 +* `fhd` = 1920×1080 +* `qhd` = 2560×1440 +* `4k` = 3840×2160 (default) + +4K is the default because the sample is designed to give a discrete GPU +enough work to outperform the CPU backend. Smaller resolutions are useful +for quick correctness checks or slower machines. + +### How pipelining works + +The synchronous path (`--pipeline 0`) follows the classic OpenVX pattern: + +1. Copy one frame into the input image. +2. Call `vxProcessGraph(graph)` and wait for it to finish. +3. Read the output image. +4. Repeat. + +The host is idle while the GPU runs, and the GPU is idle while the host copies +the next frame. That serialization limits throughput. + +The pipelined path (`--pipeline 1`) uses the `vx_khr_pipelining` extension to +keep multiple frames in flight at the same time: + +1. Create a **ring of input/output buffer pairs** (default depth = 4). +2. Expose the RGB input and the U8 output as **graph parameters**. +3. Configure the graph for `QUEUE_AUTO` mode with `vxSetGraphScheduleConfig`. +4. Prime the pipeline by enqueuing all input and output buffers with + `vxGraphParameterEnqueueReadyRef`. +5. The executor schedules graph instances automatically as soon as a full + input/output set is available. +6. The host dequeues a finished output with `vxGraphParameterDequeueDoneRef`, + records the result, refills the matching input slot, and enqueues both slots + again. + +Because the host can prepare the next frame while the previous frame is still +running on the GPU, the CPU and GPU work in parallel and overall throughput +increases. + +### Where the performance comes from + +* **Overlap host ↔ device transfers with compute.** While frame N is processed, + the host fills frame N+1 and the device writes frame N−1. +* **Keep the GPU command queue full.** Multiple enqueued frames prevent the + device from waiting between graph executions. +* **Hide CPU preprocessing.** Color conversion and channel extraction run on + the CPU and can overlap with the previous frame's GPU convolution. + +The sample prints a per-mode aggregate checksum and reports fps. Both presets +are designed to produce identical checksums for the synchronous and pipelined +paths, giving a quick correctness check. + +### Why the heavy preset matters + +On very small inputs, a GPU backend can appear slower than the CPU backend +because the per-frame launch and data-transfer overhead dominates the actual +compute time. The heavy preset raises the default resolution to 3840×2160 +and uses a Gaussian 3x3 filter so the GPU has enough work to amortize that +overhead. On a discrete GPU this typically makes the HIP backend faster than +the CPU-only backend, and the pipelined path faster than the synchronous path. + +### Tuning tips + +* **Resolution** — Use `--resolution hd|fhd|qhd|4k`. 4K is the default for + throughput demonstration; `fhd` or `hd` are good for quick correctness checks. +* **Pipeline depth** — Use `--pipeline-depth D`. The default is 4. Smaller + values reduce latency; larger values can raise throughput when the device + needs more in-flight work. +* **Mode** — Use `--mode light` for bit-exact correctness verification and + `--mode heavy` (default) for throughput measurements. +* **Frame count** — Use `--frames N` to control run length. +* **Backend** — The sample runs on CPU-only builds (`BACKEND=CPU`) as well as + on HIP/OpenCL GPU builds. On the CPU backend the gain comes from overlapping + host-side graph scheduling with data preparation. + +### Caveats when using `vx_khr_pipelining` + +A few non-obvious behaviors caught while building these samples, borrowed from +lessons learned in the larger ADAS pipeline app (PR #1730): + +* **One graph parameter rebinding.** When you enqueue a new reference for a + graph parameter, MIVisionX rebinds exactly *one* node parameter. If a + queued image is read by several nodes, only one of those nodes sees the new + reference on subsequent frames; the others silently keep using the old one. + In both samples above, the RGB input is consumed only by `ColorConvert` + and the U8 output is produced only by the final filter node, so the + rebinding is unambiguous. + +* **`VX_NODE_PERFORMANCE` returns graph time.** Asking a node for its own + `VX_NODE_PERFORMANCE` currently returns the same figure for every node in + the graph (the whole graph's execution time). That makes it impossible to + break down where time is going *inside* a single graph. To compare stage + costs, split the work into separate graphs per stage, or profile the overall + arrangement instead. + +* **Small inputs hide GPU speed-up.** On tiny frames, the per-frame GPU launch + and data-transfer cost can dominate the compute, making the CPU backend look + faster than HIP. Both samples default to 3840×2160 and the hybrid sample adds + extra filter passes to give the GPU enough work to amortize that overhead. + +## Hybrid CPU+GPU Graph Pipelining + +A second sample that explicitly pins parts of the graph to the CPU and parts +of the graph to the GPU, then pipelines the result. This is useful when you +want to force the heavy compute onto the GPU while keeping lightweight +host-side work on the CPU. + +```shell +cd c_samples/pipelining_hybrid/ +cmake . +make + +# Synchronous hybrid CPU+GPU graph +./pipelining_hybrid --pipeline 0 + +# Pipelined hybrid CPU+GPU graph +./pipelining_hybrid --pipeline 1 +``` + +### What the hybrid sample does + +The graph is deliberately split between targets: + +* **CPU nodes:** `ColorConvert`, `ChannelExtract(Y)` +* **GPU nodes:** `Box3x3 -> Box3x3 -> Box3x3` +* **CPU nodes:** `Threshold`, `Box3x3` + +The targets are pinned with `vxSetNodeTarget(..., VX_TARGET_STRING, "CPU")` and +`vxSetNodeTarget(..., VX_TARGET_STRING, "GPU")`. On a HIP backend, `"GPU"` maps +to HIP execution, so the sample demonstrates a true CPU+HIP+CPU pipeline. + +Without pipelining the three stages run one after another for each frame. +With `QUEUE_AUTO`, the executor can schedule the CPU work for frame N+1 while +the GPU is still finishing frame N, exposing the cross-target parallelism. + +This sample runs at 4K by default. Use `--resolution hd|fhd|qhd|4k` to pick a +smaller frame size, `--pipeline-depth` to tune the number of in-flight frames, +and `--compare` to run both modes back-to-back. diff --git a/samples/c_samples/pipelining/CMakeLists.txt b/samples/c_samples/pipelining/CMakeLists.txt new file mode 100644 index 000000000..ab997e7f0 --- /dev/null +++ b/samples/c_samples/pipelining/CMakeLists.txt @@ -0,0 +1,52 @@ +################################################################################ +# +# MIT License +# +# Copyright (c) 2026 Advanced Micro Devices, Inc. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in all +# copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. +# +################################################################################ + +cmake_minimum_required(VERSION 3.10) + +# ROCM Path +if(DEFINED ENV{ROCM_PATH}) + set(ROCM_PATH $ENV{ROCM_PATH} CACHE PATH "Default ROCm installation path") +elseif(ROCM_PATH) + message("-- amd_openvx_extensions:ROCM_PATH Set -- ${ROCM_PATH}") +else() + set(ROCM_PATH /opt/rocm CACHE PATH "Default ROCm installation path") +endif() +# Set AMD Clang as default compiler +set(CMAKE_CXX_STANDARD 17) +set(CMAKE_CXX_STANDARD_REQUIRED On) +set(CMAKE_CXX_EXTENSIONS ON) +if(NOT DEFINED CMAKE_CXX_COMPILER AND EXISTS "${ROCM_PATH}/lib/llvm/bin/amdclang++") + set(CMAKE_C_COMPILER ${ROCM_PATH}/lib/llvm/bin/amdclang) + set(CMAKE_CXX_COMPILER ${ROCM_PATH}/lib/llvm/bin/amdclang++) +endif() + +project(pipelining) + +find_package(OpenCV REQUIRED) +include_directories(${ROCM_PATH}/include/mivisionx ${OpenCV_INCLUDE_DIRS}) +link_directories(${ROCM_PATH}/lib) +add_executable(pipelining pipelining.cpp) +target_link_libraries(${PROJECT_NAME} openvx ${OpenCV_LIBRARIES}) diff --git a/samples/c_samples/pipelining/pipelining.cpp b/samples/c_samples/pipelining/pipelining.cpp new file mode 100644 index 000000000..87bbd7f02 --- /dev/null +++ b/samples/c_samples/pipelining/pipelining.cpp @@ -0,0 +1,587 @@ +/* +Copyright (c) 2026 Advanced Micro Devices, Inc. All rights reserved. + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in +all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +THE SOFTWARE. +*/ + +// pipelining - Demonstrates OpenVX graph pipelining (vx_khr_pipelining) on a +// CPU+GPU mixed workload. The same vision graph is executed in two modes: +// +// --pipeline 0 : synchronous vxProcessGraph loop +// --pipeline 1 : QUEUE_AUTO pipelined enqueue/dequeue with multiple buffers +// +// By default the graph is intentionally compute-heavy (4K, Gaussian 3x3) +// so a GPU backend is faster than a CPU-only backend. A lighter preset is +// available with --mode light for quick correctness checks. +// +// A --compare mode runs both paths back-to-back and prints one table, and +// --pipeline-depth lets you tune the number of in-flight frames. +// +// Both paths produce identical per-frame output; the pipelined path shows +// higher throughput because the host can fill the next input while the GPU +// is still processing the previous frame. The heavy preset uses a single +// Gaussian 3x3 GPU node so the sync and pipelined paths remain bit-exact on +// the HIP backend (chaining two GPU Box3x3 nodes through a shared virtual +// intermediate produced non-deterministic results in pipelined mode). + +#include +#include +#include + +#include + +#include +#include +#include +#include +#include + +using namespace cv; +using namespace std; +using namespace std::chrono; + +#define ERROR_CHECK_STATUS(status) { \ + vx_status status_ = (status); \ + if (status_ != VX_SUCCESS) { \ + printf("ERROR: failed with status = (%d) at " __FILE__ "#%d\n", status_, __LINE__); \ + exit(1); \ + } \ +} + +#define ERROR_CHECK_OBJECT(obj) { \ + vx_status status_ = vxGetStatus((vx_reference)(obj)); \ + if (status_ != VX_SUCCESS) { \ + printf("ERROR: failed with status = (%d) at " __FILE__ "#%d\n", status_, __LINE__); \ + exit(1); \ + } \ +} + +static const vx_uint32 PIPELINE_DEPTH_DEFAULT = 4; +static const vx_uint32 PIPELINE_DEPTH_MIN = 2; +static const vx_uint32 PIPELINE_DEPTH_MAX = 16; +static const vx_uint32 FRAME_COUNT = 120; +static const vx_uint32 WAIT_TIMEOUT_MS = 5000; + +// Resolution presets. 4K is the default because the samples are designed to +// give a discrete GPU enough work to outperform the CPU backend. +struct ResolutionPreset { + const char *name; + vx_uint32 width; + vx_uint32 height; +}; + +static const ResolutionPreset RESOLUTION_PRESETS[] = { + { "hd", 1280, 720 }, + { "fhd", 1920, 1080 }, + { "qhd", 2560, 1440 }, + { "4k", 3840, 2160 }, +}; +static const vx_uint32 RESOLUTION_PRESET_COUNT = + sizeof(RESOLUTION_PRESETS) / sizeof(RESOLUTION_PRESETS[0]); + +static void VX_CALLBACK log_callback(vx_context context, vx_reference ref, + vx_status status, const vx_char string[]) +{ + (void)context; (void)ref; (void)status; + size_t len = strlen(string); + if (len > 0) { + printf("%s", string); + if (string[len - 1] != '\n') + printf("\n"); + fflush(stdout); + } +} + +// Synthetic input: deterministic per-frame content so both modes can be compared. +static void fill_frame(Mat &input, vx_uint32 frame_idx) +{ + // Moving diagonal gradient. The actual pattern is not important; what + // matters is that it changes every frame and is reproducible. + const int offset = static_cast(frame_idx % static_cast(input.cols)); + for (int y = 0; y < input.rows; y++) { + Vec3b *row = input.ptr(y); + for (int x = 0; x < input.cols; x++) { + int max_dim = input.rows + input.cols; + int v = max_dim ? ((x + y + offset) * 255) / max_dim : 0; + row[x] = Vec3b(static_cast(v), + static_cast(255 - v), + static_cast((v + frame_idx) & 0xFF)); + } + } +} + +// Copy an OpenCV RGB Mat into an already-locked VX image. 'image' must be a +// host-accessible (non-virtual) vx_image created by the application. +static void copy_mat_to_vx_image(vx_image image, const Mat &mat, + vx_uint32 width, vx_uint32 height) +{ + vx_rectangle_t rect = { 0, 0, width, height }; + vx_imagepatch_addressing_t addr; + addr.stride_x = 3; + addr.stride_y = static_cast(mat.step); + vx_uint8 *buffer = mat.data; + ERROR_CHECK_STATUS(vxCopyImagePatch(image, &rect, 0, &addr, + buffer, VX_WRITE_ONLY, + VX_MEMORY_TYPE_HOST)); +} + +// Map a VX U8 image and compute a simple frame checksum for comparison. +static uint64_t checksum_vx_image(vx_image image, + vx_uint32 width, vx_uint32 height) +{ + vx_rectangle_t rect = { 0, 0, width, height }; + vx_map_id map_id; + vx_imagepatch_addressing_t addr; + void *ptr = nullptr; + ERROR_CHECK_STATUS(vxMapImagePatch(image, &rect, 0, &map_id, &addr, &ptr, + VX_READ_ONLY, VX_MEMORY_TYPE_HOST, + VX_NOGAP_X)); + uint64_t sum = 0; + const uint8_t *data = static_cast(ptr); + for (vx_uint32 y = 0; y < height; y++) { + for (vx_uint32 x = 0; x < width; x++) { + sum += data[y * addr.stride_y + x * addr.stride_x]; + } + } + ERROR_CHECK_STATUS(vxUnmapImagePatch(image, map_id)); + return sum; +} + +// Build the graph. The "heavy" preset applies a Gaussian 3x3 blur, which is a +// single GPU node, so the pipelined path cannot observe the non-deterministic +// intermediate-buffer sharing that occurs when two GPU Box3x3 nodes are +// chained through a shared virtual image. The "light" preset keeps a single +// Box3x3 for fast bit-exact verification. Returns the created nodes so callers +// can expose node parameters as graph parameters for pipelined queueing. +static vx_graph build_graph(vx_context context, vx_image input, vx_image output, + vx_uint32 width, vx_uint32 height, + bool heavy, + vx_node out_nodes[4], + int *out_node_count) +{ + vx_graph graph = vxCreateGraph(context); + ERROR_CHECK_OBJECT(graph); + + vx_image yuv = vxCreateVirtualImage(graph, width, height, VX_DF_IMAGE_IYUV); + vx_image luma = vxCreateVirtualImage(graph, width, height, VX_DF_IMAGE_U8); + ERROR_CHECK_OBJECT(yuv); + ERROR_CHECK_OBJECT(luma); + + out_nodes[0] = vxColorConvertNode(graph, input, yuv); + out_nodes[1] = vxChannelExtractNode(graph, yuv, VX_CHANNEL_Y, luma); + if (heavy) { + out_nodes[2] = vxGaussian3x3Node(graph, luma, output); + } else { + out_nodes[2] = vxBox3x3Node(graph, luma, output); + } + *out_node_count = 3; + + for (int i = 0; i < *out_node_count; i++) { + ERROR_CHECK_OBJECT(out_nodes[i]); + } + + ERROR_CHECK_STATUS(vxReleaseImage(&yuv)); + ERROR_CHECK_STATUS(vxReleaseImage(&luma)); + + return graph; +} + +// Make a graph parameter out of a node parameter so it can be queued. +static void add_graph_parameter(vx_graph graph, vx_node node, vx_uint32 index) +{ + vx_parameter param = vxGetParameterByIndex(node, index); + ERROR_CHECK_OBJECT(param); + ERROR_CHECK_STATUS(vxAddParameterToGraph(graph, param)); + ERROR_CHECK_STATUS(vxReleaseParameter(¶m)); +} + +static void usage(const char *name) +{ + printf("Usage: %s [--pipeline 0|1] [--mode light|heavy] [--compare] " + "[--resolution hd|fhd|qhd|4k] [--pipeline-depth D] " + "[--frames N] [--width W] [--height H]\n", name); + printf("\n"); + printf(" --pipeline 0 Run with synchronous vxProcessGraph (default)\n"); + printf(" --pipeline 1 Run with vx_khr_pipelining QUEUE_AUTO\n"); + printf(" --compare Run both --pipeline 0 and --pipeline 1 and " + "print one table\n"); + printf(" --resolution NAME Use a preset resolution (default 4k)\n"); + printf(" hd=1280x720 fhd=1920x1080 qhd=2560x1440 " + "4k=3840x2160\n"); + printf(" --pipeline-depth D Number of in-flight frames (%u-%u, default %u)\n", + PIPELINE_DEPTH_MIN, PIPELINE_DEPTH_MAX, PIPELINE_DEPTH_DEFAULT); + printf(" --mode heavy Use Gaussian 3x3 filter (default); shows GPU speed-up\n"); + printf(" --mode light Use Box3x3 filter; faster, good for correctness\n"); + printf(" --frames N Process N frames per mode (default %u)\n", FRAME_COUNT); + printf(" --width W Override input width\n"); + printf(" --height H Override input height\n"); + printf("\n"); + printf("The sample uses a mixed CPU+GPU vision graph:\n"); + printf(" heavy: RGB -> ColorConvert -> ChannelExtract(Y) -> Gaussian3x3 -> " + "U8 output\n"); + printf(" light: RGB -> ColorConvert -> ChannelExtract(Y) -> Box3x3 -> " + "U8 output\n"); + printf("\n"); + printf("Both paths compute the same per-frame checksum; the pipelined path\n"); + printf("generally reports higher fps. 4K is the default resolution so a GPU\n"); + printf("backend has enough work to outrun the CPU backend.\n"); +} + +struct Options { + bool pipeline; + bool compare; + bool heavy; + const char *resolution; + vx_uint32 pipeline_depth; + vx_uint32 frames; + vx_uint32 width; + vx_uint32 height; +}; + +static bool set_resolution_preset(Options &opt, const char *name) +{ + for (vx_uint32 i = 0; i < RESOLUTION_PRESET_COUNT; i++) { + if (strcmp(name, RESOLUTION_PRESETS[i].name) == 0) { + opt.resolution = RESOLUTION_PRESETS[i].name; + opt.width = RESOLUTION_PRESETS[i].width; + opt.height = RESOLUTION_PRESETS[i].height; + return true; + } + } + return false; +} + +static Options parse_options(int argc, char **argv) +{ + Options opt = { false, false, true, "4k", PIPELINE_DEPTH_DEFAULT, + FRAME_COUNT, 3840, 2160 }; + for (int i = 1; i < argc; i++) { + if (strcmp(argv[i], "--help") == 0 || strcmp(argv[i], "-h") == 0) { + usage(argv[0]); + exit(0); + } else if (strcmp(argv[i], "--pipeline") == 0 && i + 1 < argc) { + opt.pipeline = (atoi(argv[++i]) != 0); + } else if (strcmp(argv[i], "--compare") == 0) { + opt.compare = true; + } else if (strcmp(argv[i], "--resolution") == 0 && i + 1 < argc) { + if (!set_resolution_preset(opt, argv[++i])) { + printf("Unknown resolution preset: %s\n", argv[i]); + usage(argv[0]); + exit(1); + } + } else if (strcmp(argv[i], "--pipeline-depth") == 0 && i + 1 < argc) { + int d = atoi(argv[++i]); + if (d < static_cast(PIPELINE_DEPTH_MIN) || + d > static_cast(PIPELINE_DEPTH_MAX)) { + printf("ERROR: --pipeline-depth must be between %u and %u\n", + PIPELINE_DEPTH_MIN, PIPELINE_DEPTH_MAX); + usage(argv[0]); + exit(1); + } + opt.pipeline_depth = static_cast(d); + } else if (strcmp(argv[i], "--mode") == 0 && i + 1 < argc) { + const char *m = argv[++i]; + if (strcmp(m, "heavy") == 0) { + opt.heavy = true; + } else if (strcmp(m, "light") == 0) { + opt.heavy = false; + } else { + printf("Unknown mode: %s (use 'light' or 'heavy')\n", m); + usage(argv[0]); + exit(1); + } + } else if (strcmp(argv[i], "--frames") == 0 && i + 1 < argc) { + opt.frames = static_cast(atoi(argv[++i])); + } else if (strcmp(argv[i], "--width") == 0 && i + 1 < argc) { + opt.width = static_cast(atoi(argv[++i])); + opt.resolution = "custom"; + } else if (strcmp(argv[i], "--height") == 0 && i + 1 < argc) { + opt.height = static_cast(atoi(argv[++i])); + opt.resolution = "custom"; + } else { + printf("Unknown option: %s\n", argv[i]); + usage(argv[0]); + exit(1); + } + } + return opt; +} + +// --------------------------------------------------------------------------- +// Synchronous path: one input, one output, vxProcessGraph per frame. +// --------------------------------------------------------------------------- +static double run_synchronous(vx_context context, + vector &checksums, + const Options &opt) +{ + vx_image input = vxCreateImage(context, opt.width, opt.height, VX_DF_IMAGE_RGB); + vx_image output = vxCreateImage(context, opt.width, opt.height, VX_DF_IMAGE_U8); + ERROR_CHECK_OBJECT(input); + ERROR_CHECK_OBJECT(output); + + vx_node nodes[4]; + int node_count = 0; + vx_graph graph = build_graph(context, input, output, opt.width, opt.height, + opt.heavy, nodes, &node_count); + + // Tell the scheduler it may use both CPU and GPU targets for different + // nodes. MIVisionX defaults to the best target per node, so no explicit + // optimizer flags are required; we keep this block to document the intent. + (void)graph; + + ERROR_CHECK_STATUS(vxVerifyGraph(graph)); + + for (int i = 0; i < node_count; i++) + ERROR_CHECK_STATUS(vxReleaseNode(&nodes[i])); + + Mat input_mat(static_cast(opt.height), static_cast(opt.width), CV_8UC3); + + auto t0 = high_resolution_clock::now(); + for (vx_uint32 f = 0; f < opt.frames; f++) { + fill_frame(input_mat, f); + copy_mat_to_vx_image(input, input_mat, opt.width, opt.height); + ERROR_CHECK_STATUS(vxProcessGraph(graph)); + checksums.push_back(checksum_vx_image(output, opt.width, opt.height)); + } + auto t1 = high_resolution_clock::now(); + + ERROR_CHECK_STATUS(vxReleaseGraph(&graph)); + ERROR_CHECK_STATUS(vxReleaseImage(&input)); + ERROR_CHECK_STATUS(vxReleaseImage(&output)); + + return duration(t1 - t0).count(); +} + +// --------------------------------------------------------------------------- +// Pipelined path: QUEUE_AUTO with a ring of input/output buffers. +// --------------------------------------------------------------------------- +static double run_pipelined(vx_context context, + vector &checksums, + const Options &opt) +{ + const vx_uint32 depth = opt.pipeline_depth; + + // Create a ring of input and output buffers. Each slot is a graph parameter. + vector inputs; + vector outputs; + inputs.reserve(depth); + outputs.reserve(depth); + for (vx_uint32 i = 0; i < depth; i++) { + vx_image in = vxCreateImage(context, opt.width, opt.height, VX_DF_IMAGE_RGB); + vx_image out = vxCreateImage(context, opt.width, opt.height, VX_DF_IMAGE_U8); + ERROR_CHECK_OBJECT(in); + ERROR_CHECK_OBJECT(out); + inputs.push_back(in); + outputs.push_back(out); + } + + // Use slot 0 as the graph's default references; the rest will be enqueued. + vx_node nodes[4]; + int node_count = 0; + vx_graph graph = build_graph(context, inputs[0], outputs[0], opt.width, + opt.height, opt.heavy, nodes, &node_count); + + // Expose the input (parameter 0) and output (parameter 1) as graph + // parameters so they can be queued. The intermediate nodes are virtual. + // We need node 0 param 0 (RGB) and node 2 param 1 (U8), whether or not + // the heavy tail is present. + add_graph_parameter(graph, nodes[0], 0); // RGB input + add_graph_parameter(graph, nodes[2], 1); // U8 output + for (int i = 0; i < node_count; i++) + ERROR_CHECK_STATUS(vxReleaseNode(&nodes[i])); + + vx_graph_parameter_queue_params_t queue_params[2]; + memset(queue_params, 0, sizeof(queue_params)); + + vector input_refs; + vector output_refs; + for (vx_uint32 i = 0; i < depth; i++) { + input_refs.push_back((vx_reference)inputs[i]); + output_refs.push_back((vx_reference)outputs[i]); + } + + queue_params[0].graph_parameter_index = 0; + queue_params[0].refs_list_size = depth; + queue_params[0].refs_list = input_refs.data(); + + queue_params[1].graph_parameter_index = 1; + queue_params[1].refs_list_size = depth; + queue_params[1].refs_list = output_refs.data(); + + ERROR_CHECK_STATUS(vxSetGraphScheduleConfig(graph, + VX_GRAPH_SCHEDULE_MODE_QUEUE_AUTO, + 2, + queue_params)); + + vx_uint32 timeout = WAIT_TIMEOUT_MS; + ERROR_CHECK_STATUS(vxSetGraphAttribute(graph, VX_GRAPH_TIMEOUT, + &timeout, sizeof(timeout))); + + ERROR_CHECK_STATUS(vxVerifyGraph(graph)); + + Mat input_mat(static_cast(opt.height), static_cast(opt.width), CV_8UC3); + + // Prime the pipeline: fill and enqueue all input slots. + for (vx_uint32 i = 0; i < depth; i++) { + fill_frame(input_mat, i); + copy_mat_to_vx_image(inputs[i], input_mat, opt.width, opt.height); + ERROR_CHECK_STATUS(vxGraphParameterEnqueueReadyRef(graph, 0, + &input_refs[i], 1)); + } + + // Enqueue all output slots. Once each input has a matching output, the + // QUEUE_AUTO executor starts scheduling graph instances. + for (vx_uint32 i = 0; i < depth; i++) { + ERROR_CHECK_STATUS(vxGraphParameterEnqueueReadyRef(graph, 1, + &output_refs[i], 1)); + } + + auto t0 = high_resolution_clock::now(); + + vx_uint32 next_input_idx = depth; + vx_uint32 completed = 0; + while (completed < opt.frames) { + // Dequeue a finished output buffer, record its checksum, then recycle + // the matching input/output pair for the next frame. + vx_reference done_out = nullptr; + vx_uint32 num_done = 0; + ERROR_CHECK_STATUS(vxGraphParameterDequeueDoneRef(graph, 1, + &done_out, 1, + &num_done)); + if (num_done == 0) + continue; + + // Find which slot was returned. + vx_uint32 slot = depth; + for (vx_uint32 i = 0; i < depth; i++) { + if (done_out == output_refs[i]) { + slot = i; + break; + } + } + if (slot >= depth) { + printf("ERROR: dequeued unknown output reference\n"); + exit(1); + } + + checksums.push_back(checksum_vx_image(outputs[slot], opt.width, opt.height)); + completed++; + + if (next_input_idx < opt.frames) { + // Prepare the next input in the slot we just freed. + fill_frame(input_mat, next_input_idx); + copy_mat_to_vx_image(inputs[slot], input_mat, opt.width, opt.height); + ERROR_CHECK_STATUS(vxGraphParameterEnqueueReadyRef(graph, 0, + &input_refs[slot], 1)); + // Re-enqueue the same output slot to receive the next result. + ERROR_CHECK_STATUS(vxGraphParameterEnqueueReadyRef(graph, 1, + &output_refs[slot], 1)); + next_input_idx++; + } + } + + auto t1 = high_resolution_clock::now(); + + ERROR_CHECK_STATUS(vxReleaseGraph(&graph)); + for (vx_uint32 i = 0; i < depth; i++) { + ERROR_CHECK_STATUS(vxReleaseImage(&inputs[i])); + ERROR_CHECK_STATUS(vxReleaseImage(&outputs[i])); + } + + return duration(t1 - t0).count(); +} + +// Helper: run one configuration and print a single result line. +static void run_one(vx_context context, const Options &opt, + bool pipelined, vector &checksums) +{ + checksums.clear(); + checksums.reserve(opt.frames); + + Options run_opt = opt; + run_opt.pipeline = pipelined; + + printf("Mode: %s, preset=%s, resolution=%s, depth=%u, frames=%u, " + "size=%ux%u\n", + run_opt.pipeline ? "pipelined" : "synchronous", + run_opt.heavy ? "heavy" : "light", + run_opt.resolution, + run_opt.pipeline_depth, + run_opt.frames, run_opt.width, run_opt.height); + + double seconds = run_opt.pipeline ? run_pipelined(context, checksums, run_opt) + : run_synchronous(context, checksums, run_opt); + + double fps = static_cast(run_opt.frames) / seconds; + uint64_t total_sum = 0; + for (uint64_t c : checksums) + total_sum += c; + + printf(" time: %.3f s\n", seconds); + printf(" fps: %.1f\n", fps); + printf(" checksum aggregate: %llu\n", + static_cast(total_sum)); +} + +// --------------------------------------------------------------------------- +// main +// --------------------------------------------------------------------------- +int main(int argc, char **argv) +{ + Options opt = parse_options(argc, argv); + + vx_context context = vxCreateContext(); + ERROR_CHECK_OBJECT(context); + vxRegisterLogCallback(context, log_callback, vx_false_e); + + // The extension is compiled into openvx when OPENVX_USE_PIPELINING is ON, + // which is the default. No runtime query is needed. + (void)context; + + vector checksums; + checksums.reserve(opt.frames); + + if (opt.compare) { + printf("Comparing synchronous vs. pipelined (%s, depth=%u)\n", + opt.heavy ? "heavy" : "light", opt.pipeline_depth); + printf("-----------------------------------------------------------\n"); + run_one(context, opt, false, checksums); + uint64_t sync_sum = 0; + for (uint64_t c : checksums) sync_sum += c; + + run_one(context, opt, true, checksums); + uint64_t pipe_sum = 0; + for (uint64_t c : checksums) pipe_sum += c; + + printf("-----------------------------------------------------------\n"); + if (sync_sum == pipe_sum) { + printf("Checksums match: %llu\n", + static_cast(sync_sum)); + } else { + printf("Checksums differ: sync=%llu pipe=%llu\n", + static_cast(sync_sum), + static_cast(pipe_sum)); + } + } else { + run_one(context, opt, opt.pipeline, checksums); + } + + ERROR_CHECK_STATUS(vxReleaseContext(&context)); + return 0; +} diff --git a/samples/c_samples/pipelining_hybrid/CMakeLists.txt b/samples/c_samples/pipelining_hybrid/CMakeLists.txt new file mode 100644 index 000000000..83783c15e --- /dev/null +++ b/samples/c_samples/pipelining_hybrid/CMakeLists.txt @@ -0,0 +1,52 @@ +################################################################################ +# +# MIT License +# +# Copyright (c) 2026 Advanced Micro Devices, Inc. +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in all +# copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. +# +################################################################################ + +cmake_minimum_required(VERSION 3.10) + +# ROCM Path +if(DEFINED ENV{ROCM_PATH}) + set(ROCM_PATH $ENV{ROCM_PATH} CACHE PATH "Default ROCm installation path") +elseif(ROCM_PATH) + message("-- amd_openvx_extensions:ROCM_PATH Set -- ${ROCM_PATH}") +else() + set(ROCM_PATH /opt/rocm CACHE PATH "Default ROCm installation path") +endif() +# Set AMD Clang as default compiler +set(CMAKE_CXX_STANDARD 17) +set(CMAKE_CXX_STANDARD_REQUIRED On) +set(CMAKE_CXX_EXTENSIONS ON) +if(NOT DEFINED CMAKE_CXX_COMPILER AND EXISTS "${ROCM_PATH}/lib/llvm/bin/amdclang++") + set(CMAKE_C_COMPILER ${ROCM_PATH}/lib/llvm/bin/amdclang) + set(CMAKE_CXX_COMPILER ${ROCM_PATH}/lib/llvm/bin/amdclang++) +endif() + +project(pipelining_hybrid) + +find_package(OpenCV REQUIRED) +include_directories(${ROCM_PATH}/include/mivisionx ${OpenCV_INCLUDE_DIRS}) +link_directories(${ROCM_PATH}/lib) +add_executable(pipelining_hybrid pipelining_hybrid.cpp) +target_link_libraries(${PROJECT_NAME} openvx ${OpenCV_LIBRARIES}) diff --git a/samples/c_samples/pipelining_hybrid/pipelining_hybrid.cpp b/samples/c_samples/pipelining_hybrid/pipelining_hybrid.cpp new file mode 100644 index 000000000..c0ad82a3c --- /dev/null +++ b/samples/c_samples/pipelining_hybrid/pipelining_hybrid.cpp @@ -0,0 +1,570 @@ +/* +Copyright (c) 2026 Advanced Micro Devices, Inc. All rights reserved. + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in +all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +THE SOFTWARE. +*/ + +// pipelining_hybrid - Demonstrates a deliberately hybrid CPU+GPU OpenVX graph +// that is then pipelined with the vx_khr_pipelining extension. +// +// The graph is a three-stage chain: +// CPU: ColorConvert + ChannelExtract(Y) +// GPU: Box3x3 -> Box3x3 -> Box3x3 +// CPU: Threshold (U8) -> Box3x3 +// +// Pinning the heavy filter chain to the GPU ("GPU" maps to HIP on a HIP +// backend) and adding a final CPU stage makes the cross-target hand-off +// explicit. Without pipelining the three stages run serially per frame; +// with QUEUE_AUTO the CPU can prepare frame N+1 while the GPU filters frame N, +// and another CPU thread can finish frame N-1. +// +// Usage: +// ./pipelining_hybrid --pipeline 0 # synchronous baseline +// ./pipelining_hybrid --pipeline 1 # pipelined hybrid CPU+GPU+CPU +// ./pipelining_hybrid --compare # run both and print one table +// ./pipelining_hybrid --pipeline-depth 2 # tune in-flight frames + +#include +#include +#include + +#include + +#include +#include +#include +#include +#include + +using namespace cv; +using namespace std; +using namespace std::chrono; + +#define ERROR_CHECK_STATUS(status) { \ + vx_status status_ = (status); \ + if (status_ != VX_SUCCESS) { \ + printf("ERROR: failed with status = (%d) at " __FILE__ "#%d\n", status_, __LINE__); \ + exit(1); \ + } \ +} + +#define ERROR_CHECK_OBJECT(obj) { \ + vx_status status_ = vxGetStatus((vx_reference)(obj)); \ + if (status_ != VX_SUCCESS) { \ + printf("ERROR: failed with status = (%d) at " __FILE__ "#%d\n", status_, __LINE__); \ + exit(1); \ + } \ +} + +static const vx_uint32 PIPELINE_DEPTH_DEFAULT = 4; +static const vx_uint32 PIPELINE_DEPTH_MIN = 2; +static const vx_uint32 PIPELINE_DEPTH_MAX = 16; +static const vx_uint32 FRAME_COUNT = 120; +static const vx_uint32 WAIT_TIMEOUT_MS = 5000; + +struct ResolutionPreset { + const char *name; + vx_uint32 width; + vx_uint32 height; +}; + +static const ResolutionPreset RESOLUTION_PRESETS[] = { + { "hd", 1280, 720 }, + { "fhd", 1920, 1080 }, + { "qhd", 2560, 1440 }, + { "4k", 3840, 2160 }, +}; +static const vx_uint32 RESOLUTION_PRESET_COUNT = + sizeof(RESOLUTION_PRESETS) / sizeof(RESOLUTION_PRESETS[0]); + +static void VX_CALLBACK log_callback(vx_context context, vx_reference ref, + vx_status status, const vx_char string[]) +{ + (void)context; (void)ref; (void)status; + size_t len = strlen(string); + if (len > 0) { + printf("%s", string); + if (string[len - 1] != '\n') + printf("\n"); + fflush(stdout); + } +} + +static void fill_frame(Mat &input, vx_uint32 frame_idx) +{ + const int offset = static_cast(frame_idx % static_cast(input.cols)); + for (int y = 0; y < input.rows; y++) { + Vec3b *row = input.ptr(y); + for (int x = 0; x < input.cols; x++) { + int max_dim = input.rows + input.cols; + int v = max_dim ? ((x + y + offset) * 255) / max_dim : 0; + row[x] = Vec3b(static_cast(v), + static_cast(255 - v), + static_cast((v + frame_idx) & 0xFF)); + } + } +} + +static void copy_mat_to_vx_image(vx_image image, const Mat &mat, + vx_uint32 width, vx_uint32 height) +{ + vx_rectangle_t rect = { 0, 0, width, height }; + vx_imagepatch_addressing_t addr; + addr.stride_x = 3; + addr.stride_y = static_cast(mat.step); + vx_uint8 *buffer = mat.data; + ERROR_CHECK_STATUS(vxCopyImagePatch(image, &rect, 0, &addr, + buffer, VX_WRITE_ONLY, + VX_MEMORY_TYPE_HOST)); +} + +static uint64_t checksum_vx_image(vx_image image, + vx_uint32 width, vx_uint32 height) +{ + vx_rectangle_t rect = { 0, 0, width, height }; + vx_map_id map_id; + vx_imagepatch_addressing_t addr; + void *ptr = nullptr; + ERROR_CHECK_STATUS(vxMapImagePatch(image, &rect, 0, &map_id, &addr, &ptr, + VX_READ_ONLY, VX_MEMORY_TYPE_HOST, + VX_NOGAP_X)); + uint64_t sum = 0; + const uint8_t *data = static_cast(ptr); + for (vx_uint32 y = 0; y < height; y++) { + for (vx_uint32 x = 0; x < width; x++) { + sum += data[y * addr.stride_y + x * addr.stride_x]; + } + } + ERROR_CHECK_STATUS(vxUnmapImagePatch(image, map_id)); + return sum; +} + +static void add_graph_parameter(vx_graph graph, vx_node node, vx_uint32 index) +{ + vx_parameter param = vxGetParameterByIndex(node, index); + ERROR_CHECK_OBJECT(param); + ERROR_CHECK_STATUS(vxAddParameterToGraph(graph, param)); + ERROR_CHECK_STATUS(vxReleaseParameter(¶m)); +} + +// Build a hybrid 3-stage graph and pin each stage to CPU or GPU. The string +// targets are interpreted by MIVisionX: "GPU" selects the GPU backend (HIP or +// OpenCL), "CPU" selects the CPU implementation. +static vx_graph build_hybrid_graph(vx_context context, + vx_image input, vx_image output, + vx_uint32 width, vx_uint32 height, + vx_node out_nodes[7], + int *out_node_count) +{ + vx_graph graph = vxCreateGraph(context); + ERROR_CHECK_OBJECT(graph); + + vx_image yuv = vxCreateVirtualImage(graph, width, height, VX_DF_IMAGE_IYUV); + vx_image luma = vxCreateVirtualImage(graph, width, height, VX_DF_IMAGE_U8); + vx_image gpu1 = vxCreateVirtualImage(graph, width, height, VX_DF_IMAGE_U8); + vx_image gpu2 = vxCreateVirtualImage(graph, width, height, VX_DF_IMAGE_U8); + vx_image mask = vxCreateVirtualImage(graph, width, height, VX_DF_IMAGE_U8); + vx_image tmp = vxCreateVirtualImage(graph, width, height, VX_DF_IMAGE_U8); + ERROR_CHECK_OBJECT(yuv); + ERROR_CHECK_OBJECT(luma); + ERROR_CHECK_OBJECT(gpu1); + ERROR_CHECK_OBJECT(gpu2); + ERROR_CHECK_OBJECT(mask); + ERROR_CHECK_OBJECT(tmp); + + // CPU stage 0: color-space conversion and channel extraction. + out_nodes[0] = vxColorConvertNode(graph, input, yuv); + ERROR_CHECK_STATUS(vxSetNodeTarget(out_nodes[0], VX_TARGET_STRING, "CPU")); + out_nodes[1] = vxChannelExtractNode(graph, yuv, VX_CHANNEL_Y, luma); + ERROR_CHECK_STATUS(vxSetNodeTarget(out_nodes[1], VX_TARGET_STRING, "CPU")); + + // GPU stage 1: three filter passes to give the device substantial work. + out_nodes[2] = vxBox3x3Node(graph, luma, gpu1); + ERROR_CHECK_STATUS(vxSetNodeTarget(out_nodes[2], VX_TARGET_STRING, "GPU")); + out_nodes[3] = vxBox3x3Node(graph, gpu1, gpu2); + ERROR_CHECK_STATUS(vxSetNodeTarget(out_nodes[3], VX_TARGET_STRING, "GPU")); + out_nodes[4] = vxBox3x3Node(graph, gpu2, mask); + ERROR_CHECK_STATUS(vxSetNodeTarget(out_nodes[4], VX_TARGET_STRING, "GPU")); + + // CPU stage 2: final threshold + another CPU filter. + vx_threshold thresh = vxCreateThreshold(context, VX_THRESHOLD_TYPE_BINARY, + VX_TYPE_UINT8); + ERROR_CHECK_OBJECT(thresh); + vx_int32 threshold_value = 64; + ERROR_CHECK_STATUS(vxSetThresholdAttribute(thresh, VX_THRESHOLD_THRESHOLD_VALUE, + &threshold_value, + sizeof(threshold_value))); + out_nodes[5] = vxThresholdNode(graph, mask, thresh, tmp); + ERROR_CHECK_STATUS(vxSetNodeTarget(out_nodes[5], VX_TARGET_STRING, "CPU")); + out_nodes[6] = vxBox3x3Node(graph, tmp, output); + ERROR_CHECK_STATUS(vxSetNodeTarget(out_nodes[6], VX_TARGET_STRING, "CPU")); + + *out_node_count = 7; + + for (int i = 0; i < *out_node_count; i++) { + ERROR_CHECK_OBJECT(out_nodes[i]); + } + + ERROR_CHECK_STATUS(vxReleaseThreshold(&thresh)); + ERROR_CHECK_STATUS(vxReleaseImage(&yuv)); + ERROR_CHECK_STATUS(vxReleaseImage(&luma)); + ERROR_CHECK_STATUS(vxReleaseImage(&gpu1)); + ERROR_CHECK_STATUS(vxReleaseImage(&gpu2)); + ERROR_CHECK_STATUS(vxReleaseImage(&mask)); + ERROR_CHECK_STATUS(vxReleaseImage(&tmp)); + + return graph; +} + +static void usage(const char *name) +{ + printf("Usage: %s [--pipeline 0|1] [--compare] " + "[--resolution hd|fhd|qhd|4k] [--pipeline-depth D] " + "[--frames N] [--width W] [--height H]\n", name); + printf("\n"); + printf(" --pipeline 0 Run with synchronous vxProcessGraph (default)\n"); + printf(" --pipeline 1 Run with vx_khr_pipelining QUEUE_AUTO\n"); + printf(" --compare Run both --pipeline 0 and --pipeline 1 and " + "print one table\n"); + printf(" --resolution NAME Use a preset resolution (default 4k)\n"); + printf(" hd=1280x720 fhd=1920x1080 qhd=2560x1440 " + "4k=3840x2160\n"); + printf(" --pipeline-depth D Number of in-flight frames (%u-%u, default %u)\n", + PIPELINE_DEPTH_MIN, PIPELINE_DEPTH_MAX, PIPELINE_DEPTH_DEFAULT); + printf(" --frames N Process N frames per mode (default %u)\n", FRAME_COUNT); + printf(" --width W Override input width\n"); + printf(" --height H Override input height\n"); + printf("\n"); + printf("This sample deliberately builds a three-stage hybrid CPU+GPU+CPU\n"); + printf("graph and pins each stage:\n"); + printf(" CPU: ColorConvert, ChannelExtract(Y)\n"); + printf(" GPU: Box3x3 -> Box3x3 -> Box3x3\n"); + printf(" CPU: Threshold, Box3x3\n"); + printf("\n"); + printf("Pinning the heavy filter chain to the GPU and then pipelining the\n"); + printf("graph makes the CPU-to-GPU hand-offs explicit. The GPU filters\n"); + printf("frame N while the CPU prepares frame N+1 and finishes frame N-1.\n"); + printf("4K is the default resolution so the GPU has enough work to amortize\n"); + printf("per-frame launch/transfer overhead.\n"); +} + +struct Options { + bool pipeline; + bool compare; + const char *resolution; + vx_uint32 pipeline_depth; + vx_uint32 frames; + vx_uint32 width; + vx_uint32 height; +}; + +static bool set_resolution_preset(Options &opt, const char *name) +{ + for (vx_uint32 i = 0; i < RESOLUTION_PRESET_COUNT; i++) { + if (strcmp(name, RESOLUTION_PRESETS[i].name) == 0) { + opt.resolution = RESOLUTION_PRESETS[i].name; + opt.width = RESOLUTION_PRESETS[i].width; + opt.height = RESOLUTION_PRESETS[i].height; + return true; + } + } + return false; +} + +static Options parse_options(int argc, char **argv) +{ + Options opt = { false, false, "4k", PIPELINE_DEPTH_DEFAULT, + FRAME_COUNT, 3840, 2160 }; + for (int i = 1; i < argc; i++) { + if (strcmp(argv[i], "--help") == 0 || strcmp(argv[i], "-h") == 0) { + usage(argv[0]); + exit(0); + } else if (strcmp(argv[i], "--pipeline") == 0 && i + 1 < argc) { + opt.pipeline = (atoi(argv[++i]) != 0); + } else if (strcmp(argv[i], "--compare") == 0) { + opt.compare = true; + } else if (strcmp(argv[i], "--resolution") == 0 && i + 1 < argc) { + if (!set_resolution_preset(opt, argv[++i])) { + printf("Unknown resolution preset: %s\n", argv[i]); + usage(argv[0]); + exit(1); + } + } else if (strcmp(argv[i], "--pipeline-depth") == 0 && i + 1 < argc) { + int d = atoi(argv[++i]); + if (d < static_cast(PIPELINE_DEPTH_MIN) || + d > static_cast(PIPELINE_DEPTH_MAX)) { + printf("ERROR: --pipeline-depth must be between %u and %u\n", + PIPELINE_DEPTH_MIN, PIPELINE_DEPTH_MAX); + usage(argv[0]); + exit(1); + } + opt.pipeline_depth = static_cast(d); + } else if (strcmp(argv[i], "--frames") == 0 && i + 1 < argc) { + opt.frames = static_cast(atoi(argv[++i])); + } else if (strcmp(argv[i], "--width") == 0 && i + 1 < argc) { + opt.width = static_cast(atoi(argv[++i])); + opt.resolution = "custom"; + } else if (strcmp(argv[i], "--height") == 0 && i + 1 < argc) { + opt.height = static_cast(atoi(argv[++i])); + opt.resolution = "custom"; + } else { + printf("Unknown option: %s\n", argv[i]); + usage(argv[0]); + exit(1); + } + } + return opt; +} + +// --------------------------------------------------------------------------- +// Synchronous path. +// --------------------------------------------------------------------------- +static double run_synchronous(vx_context context, + vector &checksums, + const Options &opt) +{ + vx_image input = vxCreateImage(context, opt.width, opt.height, VX_DF_IMAGE_RGB); + vx_image output = vxCreateImage(context, opt.width, opt.height, VX_DF_IMAGE_U8); + ERROR_CHECK_OBJECT(input); + ERROR_CHECK_OBJECT(output); + + vx_node nodes[7]; + int node_count = 0; + vx_graph graph = build_hybrid_graph(context, input, output, opt.width, + opt.height, nodes, &node_count); + ERROR_CHECK_STATUS(vxVerifyGraph(graph)); + for (int i = 0; i < node_count; i++) + ERROR_CHECK_STATUS(vxReleaseNode(&nodes[i])); + + Mat input_mat(static_cast(opt.height), static_cast(opt.width), CV_8UC3); + + auto t0 = high_resolution_clock::now(); + for (vx_uint32 f = 0; f < opt.frames; f++) { + fill_frame(input_mat, f); + copy_mat_to_vx_image(input, input_mat, opt.width, opt.height); + ERROR_CHECK_STATUS(vxProcessGraph(graph)); + checksums.push_back(checksum_vx_image(output, opt.width, opt.height)); + } + auto t1 = high_resolution_clock::now(); + + ERROR_CHECK_STATUS(vxReleaseGraph(&graph)); + ERROR_CHECK_STATUS(vxReleaseImage(&input)); + ERROR_CHECK_STATUS(vxReleaseImage(&output)); + + return duration(t1 - t0).count(); +} + +// --------------------------------------------------------------------------- +// Pipelined path with QUEUE_AUTO. +// --------------------------------------------------------------------------- +static double run_pipelined(vx_context context, + vector &checksums, + const Options &opt) +{ + const vx_uint32 depth = opt.pipeline_depth; + + vector inputs; + vector outputs; + inputs.reserve(depth); + outputs.reserve(depth); + for (vx_uint32 i = 0; i < depth; i++) { + vx_image in = vxCreateImage(context, opt.width, opt.height, VX_DF_IMAGE_RGB); + vx_image out = vxCreateImage(context, opt.width, opt.height, VX_DF_IMAGE_U8); + ERROR_CHECK_OBJECT(in); + ERROR_CHECK_OBJECT(out); + inputs.push_back(in); + outputs.push_back(out); + } + + vx_node nodes[7]; + int node_count = 0; + vx_graph graph = build_hybrid_graph(context, inputs[0], outputs[0], + opt.width, opt.height, nodes, + &node_count); + + // Graph parameters: input is node 0 param 0; output is node 6 param 1. + add_graph_parameter(graph, nodes[0], 0); + add_graph_parameter(graph, nodes[6], 1); + for (int i = 0; i < node_count; i++) + ERROR_CHECK_STATUS(vxReleaseNode(&nodes[i])); + + vx_graph_parameter_queue_params_t queue_params[2]; + memset(queue_params, 0, sizeof(queue_params)); + + vector input_refs; + vector output_refs; + for (vx_uint32 i = 0; i < depth; i++) { + input_refs.push_back((vx_reference)inputs[i]); + output_refs.push_back((vx_reference)outputs[i]); + } + + queue_params[0].graph_parameter_index = 0; + queue_params[0].refs_list_size = depth; + queue_params[0].refs_list = input_refs.data(); + + queue_params[1].graph_parameter_index = 1; + queue_params[1].refs_list_size = depth; + queue_params[1].refs_list = output_refs.data(); + + ERROR_CHECK_STATUS(vxSetGraphScheduleConfig(graph, + VX_GRAPH_SCHEDULE_MODE_QUEUE_AUTO, + 2, + queue_params)); + + vx_uint32 timeout = WAIT_TIMEOUT_MS; + ERROR_CHECK_STATUS(vxSetGraphAttribute(graph, VX_GRAPH_TIMEOUT, + &timeout, sizeof(timeout))); + + ERROR_CHECK_STATUS(vxVerifyGraph(graph)); + + Mat input_mat(static_cast(opt.height), static_cast(opt.width), CV_8UC3); + + for (vx_uint32 i = 0; i < depth; i++) { + fill_frame(input_mat, i); + copy_mat_to_vx_image(inputs[i], input_mat, opt.width, opt.height); + ERROR_CHECK_STATUS(vxGraphParameterEnqueueReadyRef(graph, 0, + &input_refs[i], 1)); + } + for (vx_uint32 i = 0; i < depth; i++) { + ERROR_CHECK_STATUS(vxGraphParameterEnqueueReadyRef(graph, 1, + &output_refs[i], 1)); + } + + auto t0 = high_resolution_clock::now(); + + vx_uint32 next_input_idx = depth; + vx_uint32 completed = 0; + while (completed < opt.frames) { + vx_reference done_out = nullptr; + vx_uint32 num_done = 0; + ERROR_CHECK_STATUS(vxGraphParameterDequeueDoneRef(graph, 1, + &done_out, 1, + &num_done)); + if (num_done == 0) + continue; + + vx_uint32 slot = depth; + for (vx_uint32 i = 0; i < depth; i++) { + if (done_out == output_refs[i]) { + slot = i; + break; + } + } + if (slot >= depth) { + printf("ERROR: dequeued unknown output reference\n"); + exit(1); + } + + checksums.push_back(checksum_vx_image(outputs[slot], opt.width, opt.height)); + completed++; + + if (next_input_idx < opt.frames) { + fill_frame(input_mat, next_input_idx); + copy_mat_to_vx_image(inputs[slot], input_mat, opt.width, opt.height); + ERROR_CHECK_STATUS(vxGraphParameterEnqueueReadyRef(graph, 0, + &input_refs[slot], 1)); + ERROR_CHECK_STATUS(vxGraphParameterEnqueueReadyRef(graph, 1, + &output_refs[slot], 1)); + next_input_idx++; + } + } + + auto t1 = high_resolution_clock::now(); + + ERROR_CHECK_STATUS(vxReleaseGraph(&graph)); + for (vx_uint32 i = 0; i < depth; i++) { + ERROR_CHECK_STATUS(vxReleaseImage(&inputs[i])); + ERROR_CHECK_STATUS(vxReleaseImage(&outputs[i])); + } + + return duration(t1 - t0).count(); +} + +// Helper: run one configuration and print a single result line. +static void run_one(vx_context context, const Options &opt, + bool pipelined, vector &checksums) +{ + checksums.clear(); + checksums.reserve(opt.frames); + + Options run_opt = opt; + run_opt.pipeline = pipelined; + + printf("Mode: %s, resolution=%s, depth=%u, frames=%u, size=%ux%u\n", + run_opt.pipeline ? "pipelined" : "synchronous", + run_opt.resolution, + run_opt.pipeline_depth, + run_opt.frames, run_opt.width, run_opt.height); + + double seconds = run_opt.pipeline ? run_pipelined(context, checksums, run_opt) + : run_synchronous(context, checksums, run_opt); + + double fps = static_cast(run_opt.frames) / seconds; + uint64_t total_sum = 0; + for (uint64_t c : checksums) + total_sum += c; + + printf(" time: %.3f s\n", seconds); + printf(" fps: %.1f\n", fps); + printf(" checksum aggregate: %llu\n", + static_cast(total_sum)); +} + +// --------------------------------------------------------------------------- +// main +// --------------------------------------------------------------------------- +int main(int argc, char **argv) +{ + Options opt = parse_options(argc, argv); + + vx_context context = vxCreateContext(); + ERROR_CHECK_OBJECT(context); + vxRegisterLogCallback(context, log_callback, vx_false_e); + + vector checksums; + checksums.reserve(opt.frames); + + if (opt.compare) { + printf("Comparing synchronous vs. pipelined hybrid CPU+GPU+CPU (depth=%u)\n", + opt.pipeline_depth); + printf("-----------------------------------------------------------\n"); + run_one(context, opt, false, checksums); + uint64_t sync_sum = 0; + for (uint64_t c : checksums) sync_sum += c; + + run_one(context, opt, true, checksums); + uint64_t pipe_sum = 0; + for (uint64_t c : checksums) pipe_sum += c; + + printf("-----------------------------------------------------------\n"); + if (sync_sum == pipe_sum) { + printf("Checksums match: %llu\n", + static_cast(sync_sum)); + } else { + printf("Checksums differ: sync=%llu pipe=%llu\n", + static_cast(sync_sum), + static_cast(pipe_sum)); + } + } else { + run_one(context, opt, opt.pipeline, checksums); + } + + ERROR_CHECK_STATUS(vxReleaseContext(&context)); + return 0; +} diff --git a/utilities/runvx/gdf/pipelining/README.md b/utilities/runvx/gdf/pipelining/README.md new file mode 100644 index 000000000..333ab47f7 --- /dev/null +++ b/utilities/runvx/gdf/pipelining/README.md @@ -0,0 +1,26 @@ +# Proposed pipelining GDF examples + +These GDF files show what `vx_khr_pipelining` support in `runvx` could look like. +They use **proposed syntax** that is not yet implemented. Do not run them with +`runvx` until issue #1734 is resolved. + +## Proposed commands + +```gdf +graph parameter node param +graph schedule queue-auto depth +graph enqueue [ ...] +graph launch-pipelined frames +``` + +## Files + +- `pipelining_box3x3.gdf` — light preset: single `Box3x3` filter. +- `pipelining_gaussian3x3.gdf` — heavy preset: single `Gaussian3x3` filter. +- `pipelining_hybrid.gdf` — explicit CPU/GPU/CPU pinning with `Box3x3` chain. + +## What they test + +Once issue #1734 is implemented, these GDFs should produce deterministic +aggregate checksums across sync and pipelined runs, matching the behavior of +the C++ samples in `samples/c_samples/pipelining/`. diff --git a/utilities/runvx/gdf/pipelining/pipelining_box3x3.gdf b/utilities/runvx/gdf/pipelining/pipelining_box3x3.gdf new file mode 100644 index 000000000..d8ef852c5 --- /dev/null +++ b/utilities/runvx/gdf/pipelining/pipelining_box3x3.gdf @@ -0,0 +1,43 @@ +# Proposed pipelining GDF for the "light" C++ sample preset. +# +# Intended runvx invocation (once issue #1734 is implemented): +# runvx -frames:120 file pipelining_box3x3.gdf +# +# Graph: +# RGB input -> ColorConvert -> IYUV -> ChannelExtract(Y) -> Box3x3 -> U8 output + +set verbose off +set dump-profile off + +# Input and output are non-virtual so they can be exposed as graph parameters. +data input_0 = image:3840,2160,RGB2 +data input_1 = image:3840,2160,RGB2 +data input_2 = image:3840,2160,RGB2 +data input_3 = image:3840,2160,RGB2 + +data output_0 = image:3840,2160,U008 +data output_1 = image:3840,2160,U008 +data output_2 = image:3840,2160,U008 +data output_3 = image:3840,2160,U008 + +# Virtual intermediates inside the graph. +data yuv = virtual-image:3840,2160,IYUV +data luma = virtual-image:3840,2160,U008 + +node org.khronos.openvx.color_convert input_0 yuv +node org.khronos.openvx.channel_extract yuv !VX_CHANNEL_Y luma +node org.khronos.openvx.box_3x3 luma output_0 + +# Proposed: expose node parameters as graph parameters. +graph parameter input_0 node org.khronos.openvx.color_convert param 0 +graph parameter output_0 node org.khronos.openvx.box_3x3 param 1 + +# Proposed: configure QUEUE_AUTO with a 4-deep ring. +graph schedule queue-auto depth 4 + +# Proposed: prime the pipeline by enqueueing the rings. +graph enqueue input input_0 input_1 input_2 input_3 +graph enqueue output output_0 output_1 output_2 output_3 + +# Proposed: run 120 frames through the pipelined executor. +graph launch-pipelined frames 120 diff --git a/utilities/runvx/gdf/pipelining/pipelining_gaussian3x3.gdf b/utilities/runvx/gdf/pipelining/pipelining_gaussian3x3.gdf new file mode 100644 index 000000000..74c328007 --- /dev/null +++ b/utilities/runvx/gdf/pipelining/pipelining_gaussian3x3.gdf @@ -0,0 +1,36 @@ +# Proposed pipelining GDF for the "heavy" C++ sample preset. +# +# Intended runvx invocation (once issue #1734 is implemented): +# runvx -frames:120 file pipelining_gaussian3x3.gdf +# +# Graph: +# RGB input -> ColorConvert -> IYUV -> ChannelExtract(Y) -> Gaussian3x3 -> U8 output + +set verbose off +set dump-profile off + +data input_0 = image:3840,2160,RGB2 +data input_1 = image:3840,2160,RGB2 +data input_2 = image:3840,2160,RGB2 +data input_3 = image:3840,2160,RGB2 + +data output_0 = image:3840,2160,U008 +data output_1 = image:3840,2160,U008 +data output_2 = image:3840,2160,U008 +data output_3 = image:3840,2160,U008 + +data yuv = virtual-image:3840,2160,IYUV +data luma = virtual-image:3840,2160,U008 + +node org.khronos.openvx.color_convert input_0 yuv +node org.khronos.openvx.channel_extract yuv !VX_CHANNEL_Y luma +node org.khronos.openvx.gaussian_3x3 luma output_0 + +graph parameter input_0 node org.khronos.openvx.color_convert param 0 +graph parameter output_0 node org.khronos.openvx.gaussian_3x3 param 1 + +graph schedule queue-auto depth 4 +graph enqueue input input_0 input_1 input_2 input_3 +graph enqueue output output_0 output_1 output_2 output_3 + +graph launch-pipelined frames 120 diff --git a/utilities/runvx/gdf/pipelining/pipelining_hybrid.gdf b/utilities/runvx/gdf/pipelining/pipelining_hybrid.gdf new file mode 100644 index 000000000..514a1f2ea --- /dev/null +++ b/utilities/runvx/gdf/pipelining/pipelining_hybrid.gdf @@ -0,0 +1,59 @@ +# Proposed pipelining GDF matching the hybrid CPU+GPU+CPU C++ sample. +# +# Intended runvx invocation (once issue #1734 is implemented): +# runvx -frames:120 file pipelining_hybrid.gdf +# +# Graph: +# CPU: ColorConvert + ChannelExtract(Y) +# GPU: Box3x3 -> Box3x3 -> Box3x3 +# CPU: Threshold + Box3x3 + +set verbose off +set dump-profile off + +data input_0 = image:3840,2160,RGB2 +data input_1 = image:3840,2160,RGB2 +data input_2 = image:3840,2160,RGB2 +data input_3 = image:3840,2160,RGB2 + +data output_0 = image:3840,2160,U008 +data output_1 = image:3840,2160,U008 +data output_2 = image:3840,2160,U008 +data output_3 = image:3840,2160,U008 + +# Virtual intermediates. +data yuv = virtual-image:3840,2160,IYUV +data luma = virtual-image:3840,2160,U008 +data gpu1 = virtual-image:3840,2160,U008 +data gpu2 = virtual-image:3840,2160,U008 +data mask = virtual-image:3840,2160,U008 +data tmp = virtual-image:3840,2160,U008 + +node org.khronos.openvx.color_convert input_0 yuv +node org.khronos.openvx.channel_extract yuv !VX_CHANNEL_Y luma +node org.khronos.openvx.box_3x3 luma gpu1 +node org.khronos.openvx.box_3x3 gpu1 gpu2 +node org.khronos.openvx.box_3x3 gpu2 mask + +# Threshold: binary, value 64. +data thresh = threshold:BINARY,U008,U008 +init thresh 64 +node org.khronos.openvx.threshold mask thresh tmp +node org.khronos.openvx.box_3x3 tmp output_0 + +# Proposed: node target pinning via graph affinity directives. +# (Exact GDF syntax for vxSetNodeTarget is also not implemented today.) +graph node-target org.khronos.openvx.color_convert CPU +graph node-target org.khronos.openvx.channel_extract CPU +graph node-target org.khronos.openvx.box_3x3 GPU +graph node-target org.khronos.openvx.threshold CPU +graph node-target org.khronos.openvx.box_3x3 CPU + +graph parameter input_0 node org.khronos.openvx.color_convert param 0 +graph parameter output_0 node org.khronos.openvx.box_3x3 param 1 + +graph schedule queue-auto depth 4 +graph enqueue input input_0 input_1 input_2 input_3 +graph enqueue output output_0 output_1 output_2 output_3 + +graph launch-pipelined frames 120