Skip to content

[CI] Parity: discover job topology from workflow YAML - #3535

Open
ethanwee1 wants to merge 1 commit into
ROCm:developfrom
ethanwee1:ew/parity-discover-jobs-develop
Open

[CI] Parity: discover job topology from workflow YAML#3535
ethanwee1 wants to merge 1 commit into
ROCm:developfrom
ethanwee1:ew/parity-discover-jobs-develop

Conversation

@ethanwee1

@ethanwee1 ethanwee1 commented Aug 3, 2026

Copy link
Copy Markdown

Summary

  • download workflow YAML from raw.githubusercontent.com at the exact tested commit and read CUDA/ROCm num_shards values from the authoritative test matrix
  • follow nested reusable workflows and build-job matrix outputs, while retaining configured counts only as a failure fallback
  • self-heal renamed job prefixes and redirect artifact downloads to the workflow run that actually hosted reusable-workflow shards

Why parity_job_config.json remains

The workflow YAML is now authoritative for shard counts, but the JSON still supplies information that cannot yet be discovered from one resolved workflow file: the ordered workflow candidates and architecture-specific fallbacks, seed job prefixes used to locate the right family before self-healing, CUDA test-job kinds, and check-run/workflow regexes used by parity-auto.yml before dispatch. Removing the file now would require moving those hardcoded values elsewhere or separately redesigning workflow discovery and gating; it would not actually remove the configuration. A follow-up can remove the JSON once those remaining consumers derive their topology dynamically.

Test plan

  • Validate CUDA trunk default (14) and distributed (10) against live workflow source
  • Validate ROCm mi300 default (8) and sandbox inductor (4)
  • Validate nested CUDA inductor workflow resolution (2)
  • Run Python syntax and whitespace checks

Read CUDA and ROCm shard totals from workflow YAML at the tested commit, self-heal renamed job prefixes, and follow reusable-workflow artifacts to their hosting run.
if job_key in seen:
return None
seen.add(job_key)
job = jobs.get(job_key, {})

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we confirm that the job_key will have the value linux-jammy-rocm-py3_10-build and not linux-jammy-rocm-py3.10-mi350 in trunk.yml?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants