feat: add a Python driver that manifests, archives and publishes QA models - #8927
Merged
mc-nv merged 56 commits intoAug 26, 2026
Merged
Conversation
mc-nv
added
Feature
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
mc-nv
marked this pull request as ready for review
August 11, 2026 00:57
Greptile SummaryThe PR adds a Python QA-model generation driver with selective framework stages, per-model manifests, reproducible archive packaging, and Artifactory publication.
Confidence Score: 5/5The PR appears safe to merge. No blocking failure remains. Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart LR
A[Select framework stages] --> B[Generate QA models]
B --> C[Write per-model manifests]
C --> D[Update sizes and summary]
D --> E[Stream reproducible archives]
E --> F[Write archive index]
F --> G[Stream uploads to Artifactory]
Reviews (6): Last reviewed commit: "docs: explain the empty except in the pa..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
yinggeh
reviewed
Aug 11, 2026
These files predate the flake8 pre-commit hook and were never touched
since, so the hook had never run on them. Moving them into
model_generation makes pre-commit lint them for the first time.
gen_qa_reshape_models.py annotated three nested forward() methods with
Tuple but only imported List. Annotations in a class body are evaluated
when the enclosing function runs, so --libtorch with a string dtype at
2, 3 or 4 IOs raised NameError instead of generating models.
The rest are mechanical: drop the dead cwd-relative
sys.path.append("../common") in gen_qa_ort_scalar_models.py, hoist the
stray typing imports above the module assignments, rename the ambiguous
loop variable l to label_idx, and drop dead results of np_to_trt_dtype /
np_to_torch_dtype, which are pure lookups returning None on miss. Where
the unused value came from network.add_input the call is kept and only
the binding dropped, since it registers an input on the TensorRT
network. The duplicated shutil and torch imports under
FLAGS.torchvision_aoti are marked noqa: that branch runs independently
of the libtorch branch above it.
Also ignore the gen.*.sh and gen.*.cmds files the drivers emit into
their own directory, so hooks stop rewriting build artifacts.
gen_qa_model_repository cd's to its own directory and stages that directory into the framework containers, so narrowing it to model_generation left four things behind. test_util.py is imported by 14 generators but is shared with ~120 L0 tests, and resnet50_labels.txt is read by gen_qa_models.py as $TRITON_GENSRCDIR/resnet50_labels.txt. Both stay in qa/common and are now staged flat into gen_srcdir via TRITON_MDLS_SHARED_SRC, in both the docker and enroot paths. A symlink would not work: docker cp preserves symlinks verbatim and the target does not exist inside the container. gen_jetson_trt_models runs docker cp . without ever cd'ing to its own directory, so it staged whatever directory it happened to be invoked from. Pin it the same way the main driver does, and give it the same shared sources. Restore qa/common/gen_qa_model_repository as a symlink. readlink -f follows it, so the script still resolves model_generation as its base directory, and the GitLab CI repo -- which invokes it by that path both after cd'ing into qa/common and directly on the SLURM path -- needs no lockstep change. Point L0_backend_python/python_based_backends at the new location of gen_qa_pytorch_model.py.
gen_manifest.py builds and maintains a manifest.json beside each model's config.pbtxt, recording what produced the model and how large it is so a consumer can decide what to fetch without fetching it. It sits beside gen_common.py and is imported the same way by all sixteen generator scripts, which bracket their generation with two calls: snapshot_model_dirs() before, emit_manifests() after. The baseline is what makes per-script emission correct. Every stage writes into the same repositories -- qa_model_repository receives models from all four -- so a script that stamped everything it found would relabel the previous stage's models with its own image and framework. The snapshot fingerprints each model directory first, and only changed directories are written; a model a later script adds files to is re-stamped by that script. emit_manifests() never raises. A run that has already produced its models must not die over a metadata file, so problems are logged and counted; the phase-2 --update-sizes pass over the assembled tree is where gaps get caught. Sizes are written in two phases because a model directory receives files from more than one stage: the size recorded at creation is only final once every later stage has run. Phase 2 is idempotent -- size_bytes excludes manifest.json, so re-running produces a byte-identical file.
gen_qa_model_repository.py covers the same work as the shell driver of the same name -- four framework stages, each in its own container, on docker or enroot -- and adds what the shell version cannot express. Selective generation: --openvino, --onnx, --pytorch, --tensorrt or --all, instead of always building everything. Stage order stays fixed at OpenVINO -> ONNX -> PyTorch -> TensorRT whatever order the flags arrive in, because all four write into the same repositories. Every variable the shell driver reads also has a flag, so existing CI works unchanged while a single run can override any of them. --list and --dry-run make a flag combination inspectable without starting a multi-hour GPU job. --archive packs each model separately once generation finishes, into <version>-archives beside the tree rather than inside it: an archive written under the tree would be found by the next walk of it and packed into the next archive. gen_archive.py also runs standalone. Writing the stage scripts from Python rather than through a shell heredoc removes a class of bug the shell driver has. Its heredocs are unquoted, so every $VAR expands on the host unless escaped, and one does not: the PyTorch stage ships the host's PATH into the container, editor server directories and all. Host-side values are now explicit interpolations and a literal $VAR is evaluated in the container. Also fixed: the enroot branch's build directory cleanup, which never ran because bash discards a prefix assignment made to a function call. Verified by diffing the rendered stage scripts against the ones the shell driver generates: every generator invocation is byte-identical, and the ONNX stage matches completely. The remaining differences are the PATH fix, the igpu guard resolved host-side, the disabled OpenVINO generators reported rather than commented out, and the manifest size pass. Both engines were run end to end on the OpenVINO stage: 16 models, 16 manifests carrying the right runtime and framework, 16 archives in a separate folder, index complete, checksums matching, extraction identical to source, containers and volumes cleaned up.
A manifest pass over a full tree covers ~976 models and is not instantaneous, so printing only a closing summary line made a working run indistinguishable from a hung one. It now reports each model as it is stamped, with the fields a reader actually decides on: what serves the model, how big it is, and which tier that puts it in. --quiet keeps only the summary line. --summary PATH writes the whole pass as JSON -- counts, total bytes, breakdowns by size tier and by backend, and a record per model -- so CI can assert on the numbers instead of grepping log text. --summary - sends it to stdout. gen_qa_model_repository.py writes one to <tree>/manifest-summary.json at the end of a run. It sits at the root of the tree rather than inside a model, so it travels with the tree while no walk of it -- sizing, archiving or manifesting -- picks the file up. Verified: model discovery and total_bytes are unchanged with the summary present, and repeated passes still leave every manifest byte-identical.
enroot exposes GPUs through its 98-nvidia.sh hook, which keys entirely off NVIDIA_VISIBLE_DEVICES in the container environment and returns immediately when it is unset. The shell driver never sets it on the enroot path, so enroot builds ran without a GPU at all and their manifests recorded gpu: null. That is invisible for OpenVINO, which is CPU-only, but TensorRT plan files are specific to a compute capability, so a manifest that cannot name the GPU it was built against describes an artifact nobody can place. --nvidia-visible-devices now reaches both engines; the hook's own opt-out value, 'none', passes through unchanged. Verified by re-running the OpenVINO stage under enroot: the GPU block is now populated identically to the docker run, and the two manifests differ by exactly one line, container.runtime. All 15 end-to-end checks still pass.
docker cp names the copy after a destination that does not exist, so a fresh --output-dir received the model tree directly while an existing one received it under <version>/ -- and the closing status line pointed at the latter either way. Create the destination first, as the archive copy below it already does. Also warn when --output-dir is passed to the enroot stages, which build in place and silently ignored it.
A generation log showed which models were stamped but never what was stamped into them, so answering "which openvino was this built against?" meant unpacking a model and reading its manifest. Print the resolved properties once per pass, on both the library and command-line paths. Skipped for --update-sizes, which rewrites only the size fields and would otherwise describe the invocation rather than the manifests.
Archives were laid out mirroring the tree, named for the model alone and carrying only the model directory inside. Two consequences: an archive said nothing about which build produced it once detached from its folder, and unpacking one lost the repository it belonged to -- the same model name appears in more than one repository. Name each archive <repository>-<model>-<train>-<semver>[-<pipeline>][-<job>], flat in the destination, and preserve the tree-relative path inside, so unpacking over a tree root restores the model where it belongs. The CI pair is dropped when unset. Archives move from a sibling folder to <tree>/archives so they travel with the tree; every walk keys on a directory holding a config.pbtxt, which that folder does not. Uncompressed .tar by default, --compress for .tar.gz. TRITON_SEMVER was read by gen_manifest but never set, leaving triton_version null in every manifest and now missing from every archive name. Wire it from server/TRITON_VERSION, overridable with --semver.
Stage selection was the one option with no environment default, so a CI job that sets variables rather than composing a command line could not narrow the run. Add TRITON_MODELS_FRAMEWORKS, behind --frameworks like everything else. The value is verified and an unrecognised name refused: a typo that quietly generated nothing would not surface until a test suite failed on missing models, hours later and far from its cause. Backend names from config.pbtxt and the L0_* BACKENDS vocabulary resolve to the stage that builds them.
TRITON_VERSION does not mean one thing. The shell driver defaults it to the container train (gen_qa_model_repository:69, 26.07); GitLab exports it as the semver (.gitlab-ci.yml:59 sets TRITON_VERSION 2.71.0 beside NVIDIA_UPSTREAM_VERSION 26.07, and the models job does not override it). Both consumers read it as the train, so both were wrong under CI. Archive names came out <semver>-<semver> with no train at all, and every manifest CI produced recorded the semver in upstream_container_version, a field named for the train. Give each meaning a variable that only ever carries it: the train is NVIDIA_UPSTREAM_VERSION, the semver is TRITON_SEMVER, and TRITON_VERSION stands in for a missing train only -- the local case, which is the one where the driver did default it to the train. The driver exports both, so the two entry points into gen_manifest now resolve versions identically.
Retires TRITON_VERSION from model generation. It named the container train locally and the semver in CI, so every consumer reading it was wrong in one of the two environments. Each version now has a variable carrying exactly one meaning, all three read from server/build.py's DEFAULT_TRITON_VERSION_MAP -- the single place they are declared together: TRITON_CONTAINER_VERSION triton_container_version 26.08dev NVIDIA_UPSTREAM_VERSION upstream_container_version 26.07 TRITON_SEMVER release_version 2.72.0dev The tree is named for the container being built; the PyTorch and TensorRT images are tagged with the upstream train, which is what they actually are -- there is no nvcr.io/nvidia/pytorch:26.08dev-py3. Archiving moves out of the generation stages to the host, after collection. Packing is not model generation: doing it there lets a finished tree be re-archived under different naming, or after a failed upload, without regenerating a model, and keeps the generation images free of it. A packaging failure now warns instead of failing a run whose models are already correct. build.py is parsed rather than imported -- it is a build script, and importing it to read one dict would run its whole top level. A missing or restructured build.py falls back to constants and warns.
Neither was described, so the README implied framework bundles nested under a directory each were the only layout. Adds what the choice costs -- a bundle publishes only the fields its models agree on, so per-model archives carry the name, sizes and format versions that a bundle has to withhold -- and why the static stores are published flat, where the directory would hold one file. The 17-against-23 property counts are the measured ones.
CI keeps one run's output separate from the next by naming a directory after its pipeline, which the composed path had no way to express: .../RTX5880-x86-8.9/26.07-2.72.1dev/61511772/<archives> --build-id takes no default from the environment. Inheriting CI_PIPELINE_ID would mean a local run silently publishing somewhere other than where the same command published yesterday, decided by a variable the caller never mentioned.
CI keeps one run's output separate by naming a directory after its pipeline. gen_artifactory.py grew --build-id for that; the driver had no way to reach it, so a driver-initiated upload could not produce the layout CI needs.
NV_DOCKER_ARGS is a command, not a value. CI sets it to a curl against the local nvidia-docker plugin, and the shell driver's `eval` inside $( ) substitutes what that prints. Splitting the string put the word `curl` where docker expects the image name: docker run ... --label PROJECT_NAME=tritonserver curl -s 'http://...' -v ... docker: Error response from daemon: pull access denied for curl The comment asserted the two were equivalent 'for every value CI actually sets', which was the assumption that needed checking rather than stating -- the branch only fires when RUNNER_GPUS is numeric, so no local run reached it. Falls back to the explicit --runtime=nvidia form when the plugin is unreachable or prints nothing. The shell driver would carry on with no GPU flags at all, leaving a stage to fail later for a reason that looks unrelated.
The stage scripts ran with -e alone on the docker path, so the log carried their output but not the commands that produced it. Reproducing a failure meant reading the generated script to work out what had actually run. With -x the log holds the exact lines: + pip3 install 'numpy<=1.23.5' setuptools 'ml_dtypes<=0.5.4' + python3 .../gen_qa_reshape_models.py --openvino --models_dir=... Both drivers and both engines now agree. The shell driver already did this on its enroot path and not its docker one, and the Python driver inherited the split; neither looked deliberate.
The driver kept the shell driver's rule of leaving the tree in the volume when CI is set, which was right when a shared CIFS mount was how models reached the next job. Archiving and uploading read the tree from the host, so under CI the run skipped the copy, reported 'CI set: models stay in the volume, nothing to archive', and finished green having published nothing: jf rt s .../AGX-Thor-sbsa-11.0/ --props model.origin=dynamic -> 0 artifacts The skip now applies only when neither --archive nor --artifactory-upload was asked for, and --cleanup is honoured once the tree is safely out. Successful uploads log their destination rather than only the file name, so a reader can confirm from the job log that a run published, and where, without reconstructing the path from the flags.
An artifact named two different pipelines: the directory came from --artifactory-build-id, which CI sets to PARENT_CI_PIPELINE_ID, while the stamp in the name read CI_PIPELINE_ID from the environment -- the child. .../61627094/onnx/onnx-26.07-2.72.0dev-61627103.389119868.tar The pipeline component now follows the build id when one was given. The parent is what the artifact names already in Artifactory use and what the common property set records as CI_PIPELINE_ID, so the child was the odd one out, present only because the driver read the ambient variable. --artifactory-flat forwards gen_artifactory's --flat, which the driver could not reach, so the generated corpus can share the layout the static stores use rather than taking a directory per bundle.
A CI job forwarding a variable its runner does not define passes an empty string, which argparse takes literally and which overrides the default the flag was given for. The SLURM nodes do not define NVIDIA_VISIBLE_DEVICES, so the enroot stages received it blank. enroot's nvidia hook keys entirely off that value and does nothing when it is blank, so the container came up with no GPU and the PyTorch stage failed several minutes later with RuntimeError: Found no NVIDIA driver on your system which reads as a broken node rather than a missing argument. An empty GPU list is never meaningful; "none" remains the explicit opt-out.
NVIDIA_VISIBLE_DEVICES has a documented value set, and both drivers defaulted to "0" -- an index that exists on the docker runners and need not on a compute node. "all" is the documented way to ask for whatever GPUs are present, and is what the base CUDA images use. all every GPU 0,1 / GPU-... specific GPUs by index or UUID none no GPU, driver capabilities still enabled void / empty runtime behaves as runc -- neither GPUs nor capabilities Empty is meaningful in that list, which is why the previous commit's reasoning was wrong: it is not an invalid value, it is void. It still falls back to the default, because argparse cannot distinguish it from a caller forwarding a variable its runner never defined -- the accident that had the enroot stages build with no GPU. A caller wanting no GPU has "void" and "none" to say so, and both pass through untouched. Docker runs on a multi-GPU host now see every GPU where they saw only index 0.
The nvidia runtime mounts nvidia-smi and the driver libraries into any image, so the OpenVINO and ONNX stages -- plain ubuntu:22.04, generating on the CPU -- found a real GPU and recorded it. Every ONNX model carried the build host's compute capability, which reads as a portability constraint it does not have: a consumer selecting artifacts for an 8.9 device would wrongly exclude models that run anywhere. Stages now declare whether they need a device, and both runtimes honour it. PyTorch and TensorRT are unchanged; the two CPU stages lose the GPU arguments and their manifests record gpu: null. Their scripts already tolerated it -- the diagnostic nvidia-smi at the top of every stage is guarded with || true, and a real OpenVINO run confirms it: 'nvidia-smi: command not found', stage exits 0, manifest records no GPU.
The tree directory carried the container version, which meant two places
computed one name and could disagree. They did, twice: the upload container
cd'd into <job>/${TRITON_VERSION} while the driver wrote <job>/26.08dev, and
jfrog-upload.sh had the same assumption. Both failed silently -- docker creates
a missing working directory, so the loop found nothing and the job went green.
The name encodes nothing now, so nothing can disagree with it. The build
directory above already carries the job id, and every version that matters is
recorded where it is consumed: the manifests, the archive names and the upload
path. None of them reads this directory's name -- archive members are
tree-relative, so it never leaves the build host.
Removes --container-version and TRITON_CONTAINER_VERSION from the stage
environment, which no generator read.
The job definition records what was asked for, not what took effect. A line
reading --nvidia-visible-devices "${NVIDIA_VISIBLE_DEVICES}" tells a reader
nothing about whether the variable was set, and an empty value there is not an
absent one -- it means "void" to the container toolkit. Several failures in
this migration were an unset or empty inherited variable that nothing printed.
Rebuilt from the parsed values rather than echoed from argv, so environment
defaults, computed defaults and shell expansions all appear as the literal
value used. The token is redacted; it reaches the uploader through the
environment and never through argv, so only the echo needed handling.
Drops the --container-version row, which no longer exists, and adds two sections: where the tree lands and why its name is fixed, and the invocation echo with the reasoning for rebuilding it from parsed values rather than argv. Also corrects the standalone gen_archive/gen_artifactory examples, which still named the tree after a container version, and documents index.json -- worth stating that the upload consumes it rather than including it, so it and the per-bundle manifests exist only on the build host.
It omitted the build id segment the code has added since --artifactory-build-id was introduced, and said nothing about --flat dropping the framework directory.
The guide documented only the shell driver, which is unchanged and still correct. It said nothing about the Python driver beside it, so a reader had no way to find per-framework selection -- the thing that makes a one-backend change cheap to test. Notes the layout difference rather than glossing it: the Python driver writes a models/ directory inside the build directory instead of /tmp/<version>/, and selects GPUs by the container toolkit's values instead of assuming device 0. Links to qa/common/README.md for the rest.
mc-nv
force-pushed
the
mchornyi/TRI-1105/models-decompose-stored-models-in-artifactory
branch
from
August 24, 2026 21:31
bb1d78d to
93d791e
Compare
build_bundle_bytes packed the whole tar into io.BytesIO and returned raw.getvalue(), so an archive had to fit in memory before it could reach disk. A bundle is as large as the models it holds: README records onnx_model_store2 at 12.78 GiB, 76.7% of the corpus, and that bundle is the model plus the rest of onnx. Replaced with write_bundle(), which packs straight into a caller-supplied stream and returns (bytes_written, sha256) from a wrapper that hashes and counts on the way past. The wrapper sits outside the gzip layer so a compressed bundle still hashes to its compressed bytes, which is what Artifactory checks. Measured on a 200 MiB stand-in: peak Python allocation drops from 217.5 MiB (1.09x input) to effectively zero. Output is byte-identical to the old implementation for both the plain and gzip paths, so archives stay reproducible and existing checksums still match. Packing now writes as it goes, so a failure part way through would leave a truncated file where a valid archive belongs -- and the upload step trusts what it finds on disk. Packed under a .partial name and renamed once complete. --dry-run still reports archive_bytes and sha256; it packs to a discarding stream rather than skipping the work.
upload_archive took the archive as bytes, and the caller produced them with source.read_bytes(). That is one contiguous allocation the size of the archive -- up to 12.78 GiB for onnx_model_store2 -- and the Request was built once outside the retry loop, so it stayed resident for every attempt rather than just the one using it. It now takes a path and hands urllib an open file object. Two consequences worth stating: Content-Length is set explicitly. urllib cannot size a file object, so without it the PUT falls back to chunked transfer encoding. The request is built inside the retry loop, on a freshly opened handle. A file object is consumed by the attempt that sends it, so a request hoisted out of the loop -- as the bytes version safely was -- would replay an exhausted stream on every retry after the first. Verified against a local HTTP server: Content-Length present, no chunked encoding, checksum and auth headers unchanged, and a 503-503-200 sequence sends the full body on all three attempts. Peak allocation for a 64 MiB archive is 2.0 MiB.
CodeQL py/implicit-string-concatenation-in-list flagged five list entries built from adjacent string literals. All five are deliberate line wraps of one long shell command, not forgotten commas -- but that is exactly the ambiguity the rule exists to catch, and it reads the same to a person skimming the list. Joined with '+' instead. Verified no string changed: every string constant reachable from the module's code objects is identical before and after, which holds because CPython folds '"a" + "b"' to one constant at compile time just as it does adjacent literals. Sites: the two apt-get invocations, the pip3 pin line with its trailing shell comment, the nvidia-smi query, and the phase-2 banner.
CodeQL py/empty-except flagged four handlers that pass without comment. All four are best-effort by design; the rule's remedy is to say so, and a sibling handler a few lines away that already carries a comment was not flagged. - _read_root_mount_source: no procfs, or unreadable. The backing path is descriptive metadata and "" is the honest answer when it is unknown. - cuDriverGetVersion: enrichment only. The surrounding details already describe the GPU, so a failure leaves two fields at their None defaults rather than discarding the rest. - pynvml: importable but unusable -- no driver, or no permission. Leaving driver_version absent beats recording a value from cuDriverGetVersion, which reports something else. - enroot image cleanup: the image may already be gone or sit on a read-only mount, neither worth failing a run whose work has completed. Comments only; no behaviour changes.
CodeQL py/empty-except flagged the handler added with the streaming archiver: removing the .partial file after a failed pack passes silently if that removal itself fails. Same class as the four already documented, and self-inflicted -- the alert appeared on this branch only after 0784be2. Tidying up after a failure that is about to be raised anyway. A partial that cannot be removed stays behind under a name nothing consumes, which is not worth masking the original packing error with.
dmitry-tokarev-nv
left a comment
dmitry-tokarev-nv
left a comment
Contributor
There was a problem hiding this comment.
Review scoped to critical / blocking issues only; style, naming and latent-but-inactive divergences are left out deliberately.
Five findings, each reproduced against 4040f83 rather than read off the diff:
gen_qa_pytorch_model.py— the newimport gen_manifestbreaksL0_backend_python/python_based_backends, which copies that generator standalone. Unconditional L0 failure.gen_qa_model_repository.py—resolve_gpu_args()receives the raw namespace instead ofself, so the empty-to-allGPU fallback never reaches the docker args.gen_qa_model_repository.py—--clean-build-dirdefaults toTrue, enabling anrmtreeofTRITON_MDLS_BLD_DIRthat the shell driver never actually performed.gen_qa_model_repository— the output tree moved from$TRITON_VERSIONto$TRITON_CONTAINER_VERSION; deliberate, but out-of-repo consumers need to move with it or they go silently green.gen_manifest.py—BACKEND_TO_MODULEis keyed on backend names but looked up with stage keys, soframework_versionisnullfor pytorch and tensorrt.
The rest checked out: removed dead assignments have no remaining readers, the gen_ensemble_model_utils to gen_common move leaves no dangling emu. references, the four rendered stage scripts match the shell heredocs step-for-step and pass bash -n, and gen_archive.py round-trips both granularities with reproducible member ordering.
Automated review (Claude Code), verified by checkout and execution.
dmitry-tokarev-nv
approved these changes
Aug 26, 2026
dmitry-tokarev-nv
left a comment
dmitry-tokarev-nv
left a comment
Contributor
There was a problem hiding this comment.
Discussed on a call all concerns were addressed.
mc-nv
deleted the
mchornyi/TRI-1105/models-decompose-stored-models-in-artifactory
branch
August 26, 2026 20:29
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does the PR do?
Adds a Python driver for QA model generation beside the existing shell one, and gives every
generated model a manifest describing how it was built. Models can then be packed into
per-framework (or per-model) archives and published, so a consumer fetches the backend it
needs rather than the whole corpus.
The shell driver is unchanged and still works; nothing here is required to use it.
Three problems this addresses, each of which previously failed silently:
--onnx --pytorch, orTRITON_MODELS_FRAMEWORKS), and an unrecognised framework name is refused rather thanskipped — a typo that generated nothing used to surface hours later as a test failure.
manifest.jsonwith thecontainer, platform, framework version, GPU and format-specific fields (ONNX opset, IR
version, TensorRT version). Those flatten into properties an archive can be selected on.
nvidia-smiinto anyimage, so the CPU-only OpenVINO and ONNX stages probed a real GPU and stamped a compute
capability onto artifacts that do not depend on one. GPU access is now per stage.
Checklist
<commit_type>: <Title>Commit Type:
Check the conventional commit type
box here and add the label to the github PR.
Related PRs:
None in other GitHub repositories. The CI change that consumes these drivers lives in an
internal repository and is linked from the tracking issue.
Where should the reviewer start?
qa/common/gen_qa_model_repository.py— the driver. The stage table near the top is thewhole model: four stages, each with an image, a privilege level and a
needs_gpuflag.Then
qa/common/gen_manifest.pyfor what gets recorded, andqa/common/README.md, whichdocuments the flags and the reasoning behind the layout choices.
qa/common/gen_qa_model_repository(the shell driver) is worth a look to confirm thechanges there are confined to version handling.
Test plan:
Verified by running real generations rather than by inspection:
openvino(16 models),onnx(735 models),pytorch+tensorrt(197 models, 100.plan, 101.pt).manifests record
gpu: nullacross all 735 models, including those fromgen_qa_dyna_sequence_models.py, whilepytorchandtensorrtrecordcc=8.9.Before the change the same ONNX models carried
compute_capability: 8.9.manifest-summary.jsonat the tree root,archives/beside it holding the tarballs, per-bundle manifests andindex.json.pre-commiton every changed file.Not covered: the
L0_*suites have not been run against a corpus produced by the Pythondriver. The archives it produces are byte-identical in content to the shell driver's output
for the same inputs, but that equivalence is argued rather than measured.
Caveats:
correct, and failing a multi-hour run over packaging would discard them.
gen_archive.pyrun standalone has no fallback tobuild.pyfor the semver, so it canproduce an archive name with that field missing. The driver always passes it explicitly,
so this only affects direct invocation.
tensorrtbundle records nodriver_version; that image hascuda-pythonbut nopynvml.Background
The QA corpus was published as one artifact per repository directory, fetched whole over a
CIFS mount. A test run needing one backend pulled everything, and an unstable link made that
expensive to retry. Splitting the corpus into described, individually selectable archives is
what makes a partial fetch possible.
Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)
- Resolves: TRI-1105
CI (internal): [#62039930](http://tritonserver.local/ci/pipelines/62039930)