Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion dev/reference/supported-models/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -2978,7 +2978,7 @@
<h1 id="supported-models">Supported Models<a class="headerlink" href="#supported-models" title="Permanent link">&para;</a></h1>
<p>Windows ML CLI has validated a set of models for compatibility across all
Execution Providers (EPs)—see the full
<a href="https://microsoft.github.io/winml-cli/reports/model_compatibility_report.html">model compatibility report</a>.</p>
<a href="https://microsoft.github.io/winml-cli/reports/model_accuracy_report.html">Model Accuracy Report</a>.</p>
<p>winml-cli supports a wide range of model architectures and tasks. This page
lists what's validated and how to discover model support.</p>
<hr />
Expand Down
2 changes: 1 addition & 1 deletion dev/search/search_index.json

Large diffs are not rendered by default.

16 changes: 8 additions & 8 deletions reports/README.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
# Model compatibility report
# Model accuracy report

This folder hosts the **model compatibility report** published to the public
This folder hosts the **model accuracy report** published to the public
winml-cli GitHub Pages site, plus the script used to refresh it.

| File | Purpose |
| --- | --- |
| `model_compatibility_report.html` | The self-contained report page (all data embedded inline). |
| `model_accuracy_report.html` | The self-contained report page (all data embedded inline). |
| `download_report.py` | Fetches the latest report from the private artifacts repo. |
| `README.md` | This file. |

Expand All @@ -17,7 +17,7 @@ winml-cli GitHub Pages site, plus the script used to refresh it.
## Source

The report is generated in the private `gim-home/ModelKitArtifacts` repo and
lives at `e2e_model_coverage_result/model_compatibility_report.html` on its
lives at `e2e_model_coverage_result/model_accuracy_report.html` on its
`site-src` branch. The page embeds all of its data inline, so only this one
file needs to be published — no JSON or other assets are fetched at runtime.

Expand All @@ -42,7 +42,7 @@ cd ../wmlcli-ghpages/reports

Then run the script from this `reports/` folder. It uses only the Python
standard library — no project dependencies or `uv` required. By default it
overwrites `model_compatibility_report.html` next to the script:
overwrites `model_accuracy_report.html` next to the script:

```powershell
python download_report.py --account <your_gim-home_account>
Expand All @@ -57,12 +57,12 @@ fetches — it does not commit or push.
Commit and push the refreshed report on `gh-pages`:

```powershell
git add reports/model_compatibility_report.html
git commit -m "Update model compatibility report"
git add reports/model_accuracy_report.html
git commit -m "Update model accuracy report"
git push origin gh-pages
```

The report is then available at `.../reports/model_compatibility_report.html`.
The report is then available at `.../reports/model_accuracy_report.html`.
GitHub Pages redeploys automatically within a minute or two of the push.

If you used a worktree, clean it up afterwards:
Expand Down
12 changes: 6 additions & 6 deletions reports/download_report.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License.
# --------------------------------------------------------------------------
"""Fetch the model compatibility report from gim-home/ModelKitArtifacts.
"""Fetch the model accuracy report from gim-home/ModelKitArtifacts.

For Microsoft internal use only. Requires gh CLI authenticated with an account
that has access to the gim-home org.
Expand All @@ -25,14 +25,14 @@
python download_report.py --account <account> --out <path>

By default the report is written next to this script, overwriting the published
copy (reports/model_compatibility_report.html on the gh-pages branch).
copy (reports/model_accuracy_report.html on the gh-pages branch).

PUBLISHING (manual, done by a maintainer):
See README.md (co-located in this folder) for full instructions. After
fetching, commit and push the refreshed report on the gh-pages branch:

git add reports/model_compatibility_report.html
git commit -m "Update model compatibility report"
git add reports/model_accuracy_report.html
git commit -m "Update model accuracy report"
git push origin gh-pages
"""

Expand All @@ -47,7 +47,7 @@

SOURCE_REPO = "gim-home/ModelKitArtifacts"
SOURCE_BRANCH = "site-src"
SOURCE_FILE = "e2e_model_coverage_result/model_compatibility_report.html"
SOURCE_FILE = "e2e_model_coverage_result/model_accuracy_report.html"
REPORT_FILENAME = SOURCE_FILE.rsplit("/", 1)[-1]
DEFAULT_OUT = Path(__file__).resolve().parent / REPORT_FILENAME

Expand Down Expand Up @@ -122,7 +122,7 @@ def _sparse_clone(clone_url: str, dest: Path) -> bool:

def main() -> None:
parser = argparse.ArgumentParser(
description="Fetch the model compatibility report from gim-home/ModelKitArtifacts"
description="Fetch the model accuracy report from gim-home/ModelKitArtifacts"
)
parser.add_argument("--account", type=str, help="gh CLI account with access to gim-home org")
parser.add_argument(
Expand Down
Loading