Skip to content

fix(omnidreams): support native windows demo#353

Merged
gtong-nv merged 3 commits into
mainfrom
dev/gtong/windows-fix
Jun 26, 2026
Merged

fix(omnidreams): support native windows demo#353
gtong-nv merged 3 commits into
mainfrom
dev/gtong/windows-fix

Conversation

@gtong-nv

@gtong-nv gtong-nv commented Jun 25, 2026

Copy link
Copy Markdown
Collaborator

Native Windows OmniDreams Interactive Demo Fixes

Summary

  • Materialize --synthetic-scene before HUD/MJPEG scene-picker discovery so the
    Windows browser-stream path can start without pre-staged USDZ scenes.
  • Register Torch and CUDA runtime DLL directories before importing the Ludus JIT
    extension on Windows, and keep the returned handles alive for the process
    lifetime.
  • Add focused CPU tests for synthetic scene materialization and Windows DLL
    directory registration.

Why

The native Windows interactive-drive --stream-mjpeg --synthetic-scene path hit
two blockers before it could complete a lightweight smoke run:

  1. HUD/MJPEG startup discovered available scenes before the synthetic scene was
    materialized, so startup failed when the cache had no staged scenes.
  2. The Ludus CUDA extension compiled successfully, but importing the generated
    .pyd failed because Windows could not find dependent Torch/CUDA DLLs.

@copy-pr-bot

copy-pr-bot Bot commented Jun 25, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@gtong-nv gtong-nv force-pushed the dev/gtong/windows-fix branch from 3361469 to f102b98 Compare June 25, 2026 19:34
@gtong-nv gtong-nv marked this pull request as ready for review June 25, 2026 19:35
@gtong-nv gtong-nv force-pushed the dev/gtong/windows-fix branch from f102b98 to db23b12 Compare June 25, 2026 19:39
@gtong-nv

Copy link
Copy Markdown
Collaborator Author

/ok to test db23b12

@greptile-apps

greptile-apps Bot commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR fixes two blockers that prevented the native Windows interactive-drive --stream-mjpeg --synthetic-scene path from completing a smoke run: scene discovery now happens after the synthetic USDZ is materialized, and Torch/CUDA DLL directories are registered before the JIT extension is imported.

  • demo.py: _materialize_synthetic_scene_for_picker is called before _discover_scene_options in both HUD and MJPEG paths, clearing the synthetic args so the downstream backend builder treats the result as a normal archive.
  • _plugin.py: Windows DLL directory handles for Torch's lib/ and any discovered cudart64_*.dll directories are registered and kept alive at module scope; os.system(\"where cl.exe\") is replaced with subprocess.run([\"where.exe\", \"cl.exe\"]) for more reliable process isolation.
  • omnidreams_singleview.py: Adds _add_windows_cuda_dll_directories, _cudnn_link_flags, _cuda_link_flags, and import-library-from-DLL generation to support the full cuDNN/CUBLAS/NVRTC link graph on Windows; nvidia-cudnn-cu13==9.23.2.1 is pinned as a Windows-only dependency.

Confidence Score: 4/5

Safe to merge with one issue worth resolving: a CUDA DLL directory is silently skipped because the package lookup uses a name that does not match any known PyPI namespace.

The scene-picker ordering fix and the Ludus DLL registration are both correct and well-tested. The omnidreams_singleview.py changes introduce _python_package_dir("nvidia.cu13"), which does not correspond to any standard NVIDIA PyPI package namespace (nvidia-cuda-runtime-cu13 installs as nvidia.cuda_runtime, not nvidia.cu13). The lookup silently returns None, so the CUDA binary directory is never added to the DLL search path. If the required DLLs are not bundled in PyTorch's lib/, extension import will fail with an opaque DLL-not-found error on Windows.

integrations/omnidreams/omnidreams/native/omnidreams_singleview.py — specifically the _add_windows_cuda_dll_directories function and its _python_package_dir("nvidia.cu13") call.

Important Files Changed

Filename Overview
integrations/omnidreams/ludus-renderer/ludus_renderer/_ops/_plugin.py Adds Windows DLL directory registration for Torch/CUDA before JIT extension import; replaces os.system with subprocess.run for cl.exe detection; minor formatting improvements.
integrations/omnidreams/omnidreams/interactive_drive/demo.py Adds _materialize_synthetic_scene_for_picker to build the synthetic USDZ before HUD/MJPEG scene discovery, fixing Windows browser-stream startup ordering.
integrations/omnidreams/omnidreams/native/omnidreams_singleview.py Adds Windows cuDNN/CUDA DLL directory registration and import library generation from DLL exports; _python_package_dir("nvidia.cu13") references a non-existent PyPI package namespace, silently skipping CUDA binary directory registration.
integrations/omnidreams/tests/interactive_drive/test_demo_scene_selection.py Adds test coverage for _materialize_synthetic_scene_for_picker: verifies synthetic args are consumed, args.scene is set, and synthetic_scene is cleared to False.
integrations/omnidreams/tests/test_ludus_renderer_plugin.py New test verifying Windows DLL directory handles are kept alive and deduplication works correctly across multiple calls.
integrations/omnidreams/pyproject.toml Pins nvidia-cudnn-cu13==9.23.2.1 as a Windows-only dependency for the cuDNN DLL needed by the native extension.
integrations/omnidreams/omnidreams_singleview/patches/cutlass/sm120-tma-pool.patch Whitespace-only change: trailing spaces replaced with empty lines in the patch context, no semantic change to the CUTLASS TMA descriptor pool patch.
uv.lock Lock file updated to pin nvidia-cudnn-cu13==9.23.2.1 for win32 and 9.20.0.48 for Linux, with correct wheel hashes.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant CLI as interactive-drive CLI
    participant Demo as demo.py
    participant Synth as build_synthetic_scene_to_temp
    participant Disc as _discover_scene_options
    participant Plugin as _get_plugin()
    participant DLL as os.add_dll_directory

    CLI->>Demo: _run_streaming / _run_slangpy_hud(args)
    Demo->>Demo: _resolve_demo_paths(args)
    Demo->>Synth: _materialize_synthetic_scene_for_picker(args)
    Note over Demo,Synth: NEW: synthetic USDZ built before discovery
    Synth-->>Demo: scene_path (temp USDZ)
    Demo->>Demo: "args.scene = scene_path, args.synthetic_scene = False"
    Demo->>Disc: _discover_scene_options(args.scene_dir, args.scene)
    Disc-->>Demo: scene_options (includes synthetic scene)
    Demo->>Plugin: _get_plugin() [JIT compile/load]
    Plugin->>DLL: _ensure_windows_dll_directories()
    Note over Plugin,DLL: NEW: torch/lib + cudart64 dirs registered
    DLL-->>Plugin: handles kept in _dll_directory_handles[]
    Plugin-->>Demo: loaded .pyd extension
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant CLI as interactive-drive CLI
    participant Demo as demo.py
    participant Synth as build_synthetic_scene_to_temp
    participant Disc as _discover_scene_options
    participant Plugin as _get_plugin()
    participant DLL as os.add_dll_directory

    CLI->>Demo: _run_streaming / _run_slangpy_hud(args)
    Demo->>Demo: _resolve_demo_paths(args)
    Demo->>Synth: _materialize_synthetic_scene_for_picker(args)
    Note over Demo,Synth: NEW: synthetic USDZ built before discovery
    Synth-->>Demo: scene_path (temp USDZ)
    Demo->>Demo: "args.scene = scene_path, args.synthetic_scene = False"
    Demo->>Disc: _discover_scene_options(args.scene_dir, args.scene)
    Disc-->>Demo: scene_options (includes synthetic scene)
    Demo->>Plugin: _get_plugin() [JIT compile/load]
    Plugin->>DLL: _ensure_windows_dll_directories()
    Note over Plugin,DLL: NEW: torch/lib + cudart64 dirs registered
    DLL-->>Plugin: handles kept in _dll_directory_handles[]
    Plugin-->>Demo: loaded .pyd extension
Loading

Reviews (4): Last reviewed commit: "test(omnidreams): run ludus dll director..." | Re-trigger Greptile

Comment thread integrations/omnidreams/omnidreams/interactive_drive/demo.py
Comment thread integrations/omnidreams/ludus-renderer/ludus_renderer/_ops/_plugin.py Outdated
@gtong-nv gtong-nv force-pushed the dev/gtong/windows-fix branch from db23b12 to ad2ce6e Compare June 25, 2026 19:48
@gtong-nv

Copy link
Copy Markdown
Collaborator Author

/ok to test ad2ce6e

@jarcherNV jarcherNV left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@gtong-nv

Copy link
Copy Markdown
Collaborator Author

/ok to test 6160246

@gtong-nv

Copy link
Copy Markdown
Collaborator Author

/ok to test 67c881e

@gtong-nv gtong-nv enabled auto-merge June 26, 2026 04:31
@greptile-apps

greptile-apps Bot commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

Want your agent to iterate on Greptile's feedback? Try greploops.

@gtong-nv gtong-nv added this pull request to the merge queue Jun 26, 2026
Merged via the queue into main with commit 3684289 Jun 26, 2026
7 checks passed
@gtong-nv gtong-nv deleted the dev/gtong/windows-fix branch June 26, 2026 05:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants