Add native WGSL (genwgsl) shader generator with dedicated data library

[ashwinbhat]

The current glsl to wgsl pathway offered in MaterialX is not suitable for runtime systems.
There is an elaborate pipeline from MaterialX> GLSL -> SPIR-V -> WGSL -> [Browser]. The browser internally might convert WGSL -> SPIRV before creating byte code.

The transplied WGSL is verbose and difficult to read. The GLSL data structures are flattened sometimes due to the SPIR-V path.

This Introduces a C++ MaterialXGenWgsl backend with genwgsl shader target that has a a dedicated WGSL data library.

The WGSL data library is pre-transpiled using a helper script (glsl_to_wgsl.py) that uses Naga to create the shader nodes in WGSL data library

The MaterialX three.js viewer backend now uses WgslShaderGenerator. Unlike GLSLES, three.js does not support direct raw shaders. therefore we have an adapter to create Three.js TSL.

Note about AI use:
The script glsl_to_wgsl.py was authored using AI tools that uses naga and then performs cleanup of the shader code using simple lookup.
Screenshots:

WebGPU viewer is host here: https://ashwinbhat.github.io/MaterialX/index-webgpu.html

[jstone-lucasfilm]

Thanks for this substantial contribution, @ashwinbhat, and indeed this seems more promising than your previous #2983.

Before moving forward with detailed review, I think it's worthwhile to consider a simplification of this idea:

What if the GLSL-to-WGSL transformation step occurred only in GitHub CI, rather than on the developer's machine? The machine-generated WGSL files would then become a derived artifact rather than a maintained set of code in the repository. The source GLSL files would remain the single source of truth for hardware shading, CI would run the transpiler on each PR and package the results into our build artifacts, and developers not working with WGSL could rely upon CI to validate that they haven't broken the WGSL target with their changes. Since the generated files would no longer be committed, drift between the two libraries becomes impossible by construction, rather than managed through community conventions.

On the build side, we might then default MATERIALX_BUILD_GEN_WGSL to OFF, with GitHub CI providing Naga and setting it to ON, so that ordinary source builds require no Python or Rust toolchain. The hand-written portion of the library (the lib/ helpers, the image and light nodes, and the adapted BSDFs) would remain in the repository as maintained code, analogous to the small libraries of hand-written code in MSL and Slang today.

This direction would preserve what I see as the core contributions of this PR -- the native HwShaderGenerator-derived backend, the resource binding model, and the WebGPU viewer integration -- while shifting the transpiled library from maintained code to build infrastructure. Would you be open to exploring this path?

[ashwinbhat]

Thanks for this substantial contribution, @ashwinbhat, and indeed this seems more promising than your previous #2983.

Before moving forward with detailed review, I think it's worthwhile to consider a simplification of this idea:

What if the GLSL-to-WGSL transformation step occurred only in GitHub CI, rather than on the developer's machine? The machine-generated WGSL files would then become a derived artifact rather than a maintained set of code in the repository. The source GLSL files would remain the single source of truth for hardware shading, CI would run the transpiler on each PR and package the results into our build artifacts, and developers not working with WGSL could rely upon CI to validate that they haven't broken the WGSL target with their changes. Since the generated files would no longer be committed, drift between the two libraries becomes impossible by construction, rather than managed through community conventions.

On the build side, we might then default MATERIALX_BUILD_GEN_WGSL to OFF, with GitHub CI providing Naga and setting it to ON, so that ordinary source builds require no Python or Rust toolchain. The hand-written portion of the library (the lib/ helpers, the image and light nodes, and the adapted BSDFs) would remain in the repository as maintained code, analogous to the small libraries of hand-written code in MSL and Slang today.

This direction would preserve what I see as the core contributions of this PR -- the native HwShaderGenerator-derived backend, the resource binding model, and the WebGPU viewer integration -- while shifting the transpiled library from maintained code to build infrastructure. Would you be open to exploring this path?

Thanks @jstone-lucasfilm for the valuable feedback. I've removed the generated wgsl files and moved the generation to GitHub CI.

The MaterialX WebGPU viewer is hosted here: https://ashwinbhat.github.io/MaterialX/index-webgpu.html

[kwokcb]

I don't follow all the remapping from GLSL to WGSL but assume it's good.

For building:

• It would still be useful to have a script or set of instructions on how to build locally if you want to change and test anything. e.g. if I modify or add a GLSL file how do you rebuild it to test.

For release a few questions

• There is a comment about switching between web pages
  • Curious why are there 2 web pages -- seems like it's the simplest way to webpack ?
• Are there any issues with instantiating both a WebGL or WebGPU generators at the same time if desired on web ?
• Are all the generated WGSL files installed (visible to look at) in the release ?

[kwokcb]

This file is getting large and hard to read. Would it make sense to have a separate file for WebGPU specific functions.

[ashwinbhat]

I can move it into a separate file as a future refactoring step

[kwokcb]

Curious why 2 web pages ? Does it make sense to instead do a run-time switch, or allow for 2 different canvases with different associated renderers ?

[ashwinbhat]

I couldn't figure out how to get the UX and other elements to work seamlessly. We can improve this in future if the core generator is approved.

[kwokcb]

Why is this removed ? If wgsl is available should it not be run still.

[ashwinbhat]

The current validator uses glsllang that does not support wgsl syntax.
Note that the current generator was generating "glsl" code but the new one will generate "wgsl" code

[kwokcb]

Curious if it's possible to call into a shared setup from main and release ?

[ashwinbhat]

We have a config that is no Python based, so I kept it separate.

[ashwinbhat]

I don't follow all the remapping from GLSL to WGSL but assume it's good.

For building:

• It would still be useful to have a script or set of instructions on how to build locally if you want to change and test anything. e.g. if I modify or add a GLSL file how do you rebuild it to test.

I'll a document in the developer docs that adds steps/instructions and details about the remapping.

• Are there any issues with instantiating both a WebGL or WebGPU generators at the same time if desired on web ?
• Curious why are there 2 web pages -- seems like it's the simplest way to webpack ?
  WebGL and WebGPU are separate shader generators so it should work. The reason for separate pages is because the workflows are different

WebGL : THREE.WebGLRenderer + RawShaderMaterial (ESSL)
WebGPU :THREE.WebGPURenderer + TSL/NodeMaterial (WGSL)

• Are all the generated WGSL files installed (visible to look at) in the release ?

Yes they should be part of zip, there should be a way to verify this.

[ashwinbhat]

@kwokcb I added a developer doc which will be helpful to answer some of your question about local developer workflow

[jstone-lucasfilm]

Thanks for turning this around so quickly, @ashwinbhat. Moving the node fragments to CI generation is exactly the direction I was hoping for, and the WebGPU viewer looks great.

One remaining piece I'd like to resolve before diving into detailed review: the BSDF microfacet library still lands in the repository as a committed, hand-written WGSL copy of genglsl/lib/mx_microfacet*.glsl. This hardware shading library is arguably one of the most critical parts of the MaterialX project, and I'd rather not have two references that readers must navigate and keep in agreement.

The goal I'd propose is that the GLSL microfacet library remains the single source of truth, and the WGSL form is a CI-generated artifact that is never committed -- the same treatment the WGSL node fragments now get. From reading glsl_to_wgsl.py, this looks within reach: the main obstacle (WGSL's lack of overloading) is already solved for call sites via CALL_MAP, and the genuinely WGSL-specific residue in the lib is small.

Here's a proposed set of next steps to consider:

1. Extend the transpiler to also generate the lib/ helpers, so nothing under genwgsl/lib/ is committed.
2. Upstream any genuine divergences into the GLSL source, e.g. mx_orthonormal_basis has already drifted (the sign branch moved to fix a tangent-frame artifact), and that fix currently lives only in the WGSL copy.

It's worth noting that generation from GLSL makes drift like the mx_orthonormal_basis case impossible by construction. In the longer term I'd see render-comparison coverage for the WGSL target (building on our nightly MSL/OSL comparisons) as a natural validation gate, though that's future context rather than a request for this PR.

Does this seem like a reasonable path to you?

[ashwinbhat]

Thanks for turning this around so quickly, @ashwinbhat. Moving the node fragments to CI generation is exactly the direction I was hoping for, and the WebGPU viewer looks great.

One remaining piece I'd like to resolve before diving into detailed review: the BSDF microfacet library still lands in the repository as a committed, hand-written WGSL copy of genglsl/lib/mx_microfacet*.glsl. This hardware shading library is arguably one of the most critical parts of the MaterialX project, and I'd rather not have two references that readers must navigate and keep in agreement.

The goal I'd propose is that the GLSL microfacet library remains the single source of truth, and the WGSL form is a CI-generated artifact that is never committed -- the same treatment the WGSL node fragments now get. From reading glsl_to_wgsl.py, this looks within reach: the main obstacle (WGSL's lack of overloading) is already solved for call sites via CALL_MAP, and the genuinely WGSL-specific residue in the lib is small.

Here's a proposed set of next steps to consider:

1. Extend the transpiler to also generate the lib/ helpers, so nothing under genwgsl/lib/ is committed.
2. Upstream any genuine divergences into the GLSL source, e.g. mx_orthonormal_basis has already drifted (the sign branch moved to fix a tangent-frame artifact), and that fix currently lives only in the WGSL copy.

It's worth noting that generation from GLSL makes drift like the mx_orthonormal_basis case impossible by construction. In the longer term I'd see render-comparison coverage for the WGSL target (building on our nightly MSL/OSL comparisons) as a natural validation gate, though that's future context rather than a request for this PR.

Does this seem like a reasonable path to you?

hi @jstone-lucasfilm it seems reasonable but from my findings, lib/ helpers are very difficult to transpile using naga as they are missing a lot of context and other data structures that is emitted by shader generators.

I agree it makes sense update the mx_orthonormal_basis

It might be more valuable to have wgsl specific implementations of some microfacet library functions that are better authored for Web instead of porting the glsl versions as-is. Naga does create very verbose shaders that increases the overall source length.

[ashwinbhat]

Hi @jstone-lucasfilm
Based on your suggestion, I have modified the WGSL "pre-transpile" pipeline. The subset of libraries for WGSL now are math and fragments that need texture support.
The transpile is done via naga. Naga, produces "correct" but very verbose, hard to read WGSL due to static single assignment (SSA) style code with temporaries, hoisted var declarations, redundant nested blocks, mangled identifier names.
I've introduced a helper that "cleans-up" the naga transpiled code by understanding the WGSL grammar. It doesn't regenerate code, only deletes/renames/reindents etc.
Let me know if this are suitable changes and towards the direction you envision.

[jstone-lucasfilm]

Thanks for turning this around, @ashwinbhat. Generating the microfacet library from genglsl is exactly the direction I was hoping for, and the cleanup pass makes the output far more readable than raw Naga output.

Two issues I'd like to resolve before diving into detailed review:

The first is the hand-written library remaining under stdlib/genwgsl/lib, still around 1,200 lines with mx_noise alone at 716. The skip list justifies these as "whole dir maintained by hand (was HANDWRITTEN_LIB_DIRS)", which reads as a record of the tool's earlier state rather than a technical constraint, and differs in character from the pbrlib and node entries above it. Looking through the sources, only mx_math appears genuinely blocked and only in part -- the mx_mod overloads and mx_isinf need hand-writing, while mx_matrix_mul, mx_square and mx_srgb_encode are ordinary GLSL, so perhaps an mx_math_platform.wgsl split mirroring mx_shadow_platform. mx_noise, mx_flake, mx_hsv and mx_geometry have no samplers, no $-tokens and no derivatives, and mxgenwgsl.py already carries a LIB_USE_SIBLING_BODIES entry for mx_flake and mx_noise that transpileLibs can never reach, since isHandWrittenLib short-circuits first. mx_hextile I'd set aside for now. Its dFdx and dFdy calls are only a spelling difference -- MslShaderGenerator and SlangShaderGenerator already map those tokens, and we could do the same for WGSL or wrap them in mx_math alongside mx_mod and mx_isinf. But WGSL enforces uniform control flow for derivatives as a hard error rather than leaving it undefined, so that one deserves its own look.

Was the stdlib pass attempted after the lib refactor, or is this the original setting carried forward? I'd suggest dropping those four from skip_transpile.txt and running the tool: if they transpile, close to a thousand lines leave the repository, and if they don't, we learn the reason the skip list is currently missing. My concern is the one we discussed for the microfacet library -- mx_noise has 58 function definitions whose numerics must match genglsl exactly, and coverage matches today just as it did for the microfacet library right up until mx_orthonormal_basis drifted.

The second is mx_orthonormal_basis itself, where I think you've found a real bug that isn't WGSL-specific. mx_generate_prefilter_env builds its normal from the latlong projection, where N.z is cos(latitude) * cos(longitude). That crosses zero at longitude +/-90 degrees, so the N.z < 0 branch runs down two vertical lines of the environment map, with the tangent frame flipping across each one. That flip affects GLSL, ESSL, MSL and Slang today, and is latent only because FIS is our default environment method. Moving the branch to -0.9999999 hides it on the map's wrap seam rather than removing it, and it costs precision: the original sign choice exists to keep sign + N.z away from zero, and the new threshold lets it cancel in f32, so the basis stops being orthonormal near that seam. Since mx_generate_prefilter_env already has uv in hand, I'd suggest building the frame from the latlong parameterization itself -- a tangent along the longitude direction, with the bitangent from the cross product -- which is continuous across the whole map and fixes the seam for every hardware target at once.

Could we take that as its own PR against main, with before-and-after images? The same for the tf_thickness / tf_ior rename -- I'm not opposed to the names, but it's a change to a public data library and I'd rather it stood on its own than rode along as a transpiler simplification. Together those would let this PR stop modifying pbrlib/genglsl entirely.