Add Sensor PPISP controller support for camera post-processing

[moennen]

Description

Adds PPISP PPISPAuto controller support to IsaacLab camera post-processing.

This change parses embedded PPISP controller weights from USD/SPG-authored controller CUDA sources, stores them in PpispCfg, and evaluates the controller in Warp before applying PPISP.
The controller predicts per-camera exposure and color-latent parameters, while existing static PPISP config continues to provide responsivity, vignetting, and CRF parameters.

The implementation keeps the original pure-Warp controller path as a correctness reference and adds a native wp.func_native CUDA-snippet backend for high-env-count tiled camera workloads. The default controller backend uses an automatic heuristic: Warp for single/large images, native CUDA snippets for many small camera tiles.

Also adds tests for:

• USD PPISPAuto controller parsing
• controller weight layout and NumPy equivalence
• native-vs-Warp controller equivalence
• PPISP camera sensor coverage across Isaac RTX, OVRTX, and Newton paths
• opt-in PPISP pipeline performance sweep

Dependencies:

• Existing Warp dependency
• CUDA required for the native controller backend and performance test

Fixes #

Type of change

• New feature (non-breaking change which adds functionality)

Screenshots

Not applicable. This is camera post-processing and renderer/backend integration behavior.

Checklist

• I have read and understood the contribution guidelines
• I have run the pre-commit checks with ./isaaclab.sh --format
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• I have updated the changelog and the corresponding version in the extension's config/extension.toml file
• I have added my name to the CONTRIBUTORS.md or my name already exists there

[greptile-apps]

Greptile Summary

This PR adds PPISPAuto controller support to the IsaacLab PPISP camera post-processing pipeline. It migrates the USD data model from shader-prim-child attributes to direct ppisp:* attributes on camera prims, parses and stores embedded controller network weights in PpispCfg, and evaluates the controller via wp.func_native CUDA snippets before applying the per-frame exposure/color-latent predictions.

• cfg.py: replaces the old shader_prim_path model with camera_prim_path, adds controller_weights / controller_responsivity / controller_prior_exposure fields, and introduces stage-wide discovery helpers (auto_any_ppisp_cfg, resolve_and_normalize).
• kernels.py: adds a two-stage native CUDA controller (CNN pool → MLP) with weight offsets validated at import time; a separate apply_ppisp_to_rgba_with_controller_params kernel consumes per-camera exposure/color-latent params predicted by the controller.
• pipeline.py: PpispPipeline now manages per-device weight upload and per-shape scratch-buffer caches; the stage argument is removed from __init__, enforcing that normalization happens upstream in the renderer prepare_cameras hook.

Confidence Score: 4/5

Safe to merge; the controller computation is well-tested with NumPy equivalence checks and the renderer guard refactors are correct.

The implementation is thorough and the NumPy-vs-CUDA equivalence test validates the core numerical path. The only issues found are style-level: a redundant camera_prim_path = None assignment, a cache key that encodes image dimensions the buffers don't actually use (causing silent memory accumulation if resolution ever varies mid-session), and an asymmetric in-place mutation in normalize_ppisp_cfg that could surprise callers on reset paths.

pipeline.py (_controller_buffers_by_shape cache key) and cfg.py (normalize_ppisp_cfg mutation semantics)

Important Files Changed

Filename	Overview
source/isaaclab_ppisp/isaaclab_ppisp/cfg.py	Replaces shader-prim-path USD model with direct camera ppisp:* attribute parsing; adds controller weight ingestion, normalize_ppisp_cfg path-gated merge, and stage-wide discovery helpers. The asymmetric mutation behaviour in normalize_ppisp_cfg and redundant camera_prim_path = None assignments are minor issues.
source/isaaclab_ppisp/isaaclab_ppisp/kernels.py	Adds native CUDA snippet-backed CNN pool + MLP controller kernels and the corresponding compute_ppisp_controller_params / apply_ppisp_to_rgba_with_controller_params public API; weight offset constants validated at import time against the architectural constant. The NumPy reference in tests confirms correctness.
source/isaaclab_ppisp/isaaclab_ppisp/pipeline.py	Removes stage from PpispPipeline.__init__, adds per-device weight upload cache and per-shape scratch-buffer cache for the controller path. Buffer cache key over-encodes image dimensions that don't affect buffer allocation.
source/isaaclab_physx/isaaclab_physx/renderers/isaac_rtx_renderer.py	Decouples PPISP normalization from camera-path availability, allowing sentinel discovery before camera prims are registered; apply_rtx_exposure_overrides is still guarded by an explicit path check.
source/isaaclab_ov/isaaclab_ov/renderers/ovrtx_renderer.py	Same camera-path guard refactor as IsaacRtxRenderer; apply_rtx_exposure_overrides call is correctly guarded by not spec.camera_prim_paths.
source/isaaclab_newton/isaaclab_newton/renderers/newton_warp_renderer.py	Minimal guard refactor; allows resolve_and_normalize to run without camera paths, consistent with the other renderer changes.
source/isaaclab_ppisp/test/test_ppisp.py	Migrates existing tests from shader-prim to camera-attr model and adds thorough coverage: controller-weight parsing, default-override non-interference, responsivity propagation, and per-discovery-mode paths.
source/isaaclab_ppisp/test/test_ppisp_kernels.py	Adds controller-param layout tests and a full NumPy-equivalence validation for the native CUDA controller at rtol=2e-4, confirming CUDA and reference paths agree within float32 tolerance.
source/isaaclab_ppisp/test/test_ppisp_pipeline_perf.py	Opt-in performance sweep (off by default) for multi-resolution, multi-env PPISP/controller workloads; correctly guarded by ISAACLAB_PPISP_PERF=1.
source/isaaclab_ppisp/isaaclab_ppisp/_demo_utils.py	New shared helpers for demo scripts: finds PPISP cameras by ppisp:* attrs, resolves render products, and orders cameras by stage traversal order.

Sequence Diagram

sequenceDiagram
    participant Renderer as Renderer (prepare_cameras)
    participant RN as resolve_and_normalize
    participant CFG as PpispCfg / USD Camera
    participant PP as PpispPipeline (apply)
    participant CTRL as compute_ppisp_controller_params
    participant ISP as apply_ppisp_to_rgba_with_controller_params

    Renderer->>RN: isp_cfg, stage, camera_prim_path
    RN->>CFG: auto_camera_ppisp_cfg / auto_any_ppisp_cfg
    CFG-->>RN: PpispCfg (weights + inputs)
    RN-->>Renderer: normalized PpispCfg
    Renderer->>PP: PpispPipeline(cfg)
    Note over PP: caches weights on first device use
    PP->>CTRL: hdr_color, weights, features, controller_params
    Note over CTRL: CNN pool kernel (native CUDA) / MLP kernel (native CUDA)
    CTRL-->>PP: controller_params (N x 9)
    PP->>ISP: hdr_color, out_rgba, cfg, controller_params
    ISP-->>PP: LDR RGBA (in-place)

Loading

_{Reviews (1): Last reviewed commit: "Add PPISP controller changelog fragments" | Re-trigger Greptile}