SAI VPP UT Harness - PR Check Support in sonic-platform-vpp - #264
SAI VPP UT Harness - PR Check Support in sonic-platform-vpp#264nicholasching wants to merge 3 commits into
Conversation
Signed-off-by: Nicholas Ching <nicholaslching@gmail.com>
|
/azp run |
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
Signed-off-by: Nicholas Ching <nicholaslching@gmail.com>
|
/azp run |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
There was a problem hiding this comment.
Pull request overview
Adds Azure Pipelines support to validate sonic-platform-vpp VPP changes against the sonic-sairedis SAI VPP unit-test harness by building a derived test image with current-run VPP debs and executing a fixed test matrix on the sonictest pool.
Changes:
- Introduces
BuildSaiVppTestImageandTestSaiVpppipeline stages wired intoazure-pipelines.ymlwith a newsairedis_run_idparameter. - Adds templates to (1) overlay current-run VPP debs into an approved
docker-sai-test-vppbase image and (2) run/publish the SAI VPP compatibility test matrix results. - Documents the new CI flow in
README.md.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| README.md | Documents the new VPP SAI compatibility CI flow and how it sources the sairedis artifact and test policy. |
| azure-pipelines.yml | Adds a sairedis_run_id parameter and new stages to build an overlay test image and run the SAI VPP matrix. |
| .azure-pipelines/build-sai-vpp-test-image-template.yml | Implements the “overlay VPP debs onto approved sairedis image” build and provenance/checksum generation. |
| .azure-pipelines/test-sai-vpp-template.yml | Runs the matrix in a privileged container on sonictest, evaluates baseline, and publishes results/logs. |
| .azure-pipelines/Dockerfile.sai-vpp-overlay | Dockerfile for installing/reinstalling the selected VPP debs onto the approved base image. |
| - name: sairedis_run_id | ||
| type: string | ||
| default: '1170125' |
| copy_one_deb 'libvppinfra_*_amd64.deb' | ||
| copy_one_deb 'vpp_*_amd64.deb' | ||
| copy_one_deb 'vpp-plugin-core_*_amd64.deb' | ||
| copy_one_deb 'vpp-plugin-dpdk_*_amd64.deb' |
| contract_dir="$download_dir/ci-contract" | ||
| matrix_rc="$(cat "$results_dir/matrix.rc")" | ||
|
|
||
| python3 -c 'import defusedxml' 2>/dev/null || sudo uv pip install --system defusedxml |
|
All tests, including the newly added SAI VPP tests are now passing. This concludes the following features on this draft PR, which can be reviewed after the following
I have attached the results of a recent stable build below to checkpoint development: The most recent commit, pinning the sairedis artifact to a successful build on my fork, was made to validate functionality of the SAI VPP test harness. To prepare this PR for review, that change will be reverted in the next commit; therefore, the SAI VPP tests will fail until the aforementioned PR dependencies are merged to |
Signed-off-by: Nicholas Ching <nicholaslching@gmail.com>
|
/azp run |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.
Comments suppressed due to low confidence (1)
.azure-pipelines/test-sai-vpp-template.yml:89
- This step installs
defusedxmlviauv, butuvis not used elsewhere in this repo and may not be present on the self-hostedsonictestpool. Ifuvis missing, the matrix evaluation will fail even thoughpython3is available. Consider using a fallback chain (uv -> pip -> install pip) so the job is self-contained.
python3 -c 'import defusedxml' 2>/dev/null || sudo uv pip install --system defusedxml
| - publish: $(Build.ArtifactStagingDirectory)/sai-vpp-results | ||
| artifact: ${{ parameters.log_artifact_name }}@$(System.JobAttempt) | ||
| displayName: Publish VPP SAI logs and results |
Description of PR
Summary:
Phase 3 of the SAIVPP unit-test framework: adds the consumer-side CI workflow for
sonic-platform-vpp. The pipeline overlays the VPP packages built from the current pull request onto an approveddocker-sai-test-vppimage published bysonic-sairedis, then runs the same compatibility matrix and stable-baseline evaluator carried by that artifact. Reviewers can start atazure-pipelines.ymland.azure-pipelines/build-sai-vpp-test-image-template.yml.This PR depends on sonic-net/sonic-sairedis#2034. That PR is the producer of the
docker-sai-test-vppartifact, including its image tag, checksums, provenance, full-matrix selector list, stable-pass baseline, matrix generator, and evaluator underci-contract/. The default path in this PR selects the latest successfulsonic-sairedismaster artifact; artifacts produced before #2034 do not contain this contract, so the default pipeline cannot pass until #2034 merges and publishes a successful master artifact.Fixes # (N/A - no upstream issue)
Type of change
Approach
What is the motivation for this PR?
The existing pipeline verifies that VPP packages build, but it does not detect whether a VPP change regresses behavior exposed through the SAI VPP backend.
The package-fed harness introduced by sonic-net/sonic-sairedis#1950 defined a third integration use case: a
sonic-platform-vpppull request should reuse an approved sairedis test image, replace only the VPP packages with packages built from the current change, and run the same matrix and regression policy. This PR implements that workflow without rebuilding or independently versioning sonic-sairedis, SAI Thrift, the test suite, or its baseline policy.Work item tracking
How did you do it?
azure-pipelines.yml: addsBuildSaiVppTestImageafterBuildVppandTestSaiVppafter the derivative image is published. It also adds an optionalsairedis_run_idparameter; an empty value selects the latest successfulAzure.sonic-sairedismaster artifact, while an immutable build ID can be supplied for controlled validation..azure-pipelines/Dockerfile.sai-vpp-overlayand.azure-pipelines/build-sai-vpp-test-image-template.yml: load the approved sairedis image and reinstall exactly the current run'slibvppinfra,vpp,vpp-plugin-core, andvpp-plugin-dpdkpackages.libsairedis,libsaivs,libsaimetadata,saiserver, andpython-saithriftretain their original package identities.ci-contract/from sairedis. The matrix and baseline policy are therefore owned in one place rather than duplicated between repositories..azure-pipelines/test-sai-vpp-template.yml: runssai_route_test,sai_rif_test,sai_neighbor_test, andsai_ecmp_testatPORT_COUNT=32with per-test isolation. It publishes JUnit results and always archives logs, checksums, and provenance.sonictestagents can clean stale resources without affecting unrelated workloads.README.md: documents the producer/consumer workflow, default artifact selection, and immutable-run override.How did you verify/test it?
Azure build 1170172 validated the complete consumer path against immutable sonic-sairedis producer build
1170125.BuildVpp,BuildSaiVppTestImage, andTestSaiVppcompleted successfully, exercising the cross-project artifact download, checksum and contract validation, current-run VPP overlay, package-boundary checks, privileged matrix execution, stable-baseline evaluation, and evidence publication.The raw matrix retained known non-baseline failures by design; success means the producer's complete selector contract was observed and its stable baseline passed, not that every discovered test passed.
A local derivative-image smoke test also passed
sai_route_test.RouteRifTestwith one JUnit testcase and zero failures, errors, or skips while retaining the original sairedis and SAI Thrift package identities.git diff --checkpasses for this change.The hosted validation temporarily selected producer build
1170125. Commitf11df0drestored the final emptysairedis_run_iddefault afterward, so normal runs again select the latest successful sairedis master artifact.Any platform specific information?
VPP platform CI only. The matrix runs in a privileged container because the harness uses raw AF_PACKET, veth, and TAP interfaces. This PR changes CI and documentation only; it does not modify VPP production runtime or dataplane code.
Documentation
Extends
README.mdwith the VPP SAI compatibility workflow, the package-overlay boundary, artifact selection behavior, and the trusted-maintainer requirement for privileged pull-request execution.Dependencies and merge order
sonic-net/sonic-sairedis#2034 must merge first and complete a successful master pipeline run. That run publishes the image and
ci-contract/consumed by this PR's default artifact-selection path. The two PRs can be reviewed in parallel, andsairedis_run_idcan select a producer PR build for validation, but #264 should merge only after the producer artifact is available from sairedis master.