Skip to content

Add upload-llm-artifacts common template step + pipelineArtifact output#47805

Open
ReilleyMilne wants to merge 3 commits into
mainfrom
reilleymilne/test-llm-artifacts-upload
Open

Add upload-llm-artifacts common template step + pipelineArtifact output#47805
ReilleyMilne wants to merge 3 commits into
mainfrom
reilleymilne/test-llm-artifacts-upload

Conversation

@ReilleyMilne

Copy link
Copy Markdown
Member

This pull request introduces a new pipeline step to collect and upload test result files as "LLM Artifacts" to support downstream LLM tooling (such as GitHub Copilot) for test analysis. The changes add a reusable, language-agnostic template for staging test result files and update the CI test job to use this template and publish the artifacts when available.

Copilot AI review requested due to automatic review settings July 1, 2026 23:15

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR wires up collection and upload of test result files as "LLM Artifacts" so downstream LLM tooling (e.g. GitHub Copilot) can analyze CI test runs. It invokes the existing language-agnostic eng/common/pipelines/templates/steps/upload-llm-artifacts.yml staging template from the shared build-test.yml, and declares a templateContext.outputs pipeline artifact in the CI test job so the staged files are published when present.

Changes:

  • Invoke upload-llm-artifacts.yml after PublishTestResults@2 in build-test.yml, scoped to $(Build.SourcesDirectory)/sdk with the Python glob *test*.xml.
  • Add a templateContext.outputs pipelineArtifact in ci.tests.yml that publishes the llm-artifacts staging directory (per-job/attempt name), gated on uploadLlmArtifacts == 'true' with SBOM disabled.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
eng/pipelines/templates/steps/build-test.yml Adds the LLM-artifact staging template call after test-result publication; staging runs before code coverage publish.
eng/pipelines/templates/jobs/ci.tests.yml Declares the pipeline artifact output that uploads the staged llm-artifacts directory, conditioned on the runtime uploadLlmArtifacts variable.

Comment on lines +189 to +192
- template: /eng/common/pipelines/templates/steps/upload-llm-artifacts.yml
parameters:
TestResultsGlob: '*test*.xml'
SearchFolder: '$(Build.SourcesDirectory)/sdk'
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants