From 47e9a11066348f478b6712aba42cb5c8b5ff5c9e Mon Sep 17 00:00:00 2001 From: M Platypus Date: Tue, 4 Aug 2026 20:13:38 -0400 Subject: [PATCH 1/3] fix: CI never ran tests for 3 of 4 packages, or even triggered on them Multiple independent Opus peer reviews of PRs opened this session flagged the same underlying gap repeatedly: none of the new regression tests added to tinyml-tinyverse, tinyml-modelzoo, or tinyml-modeloptimization/torchmodelopt actually run in CI. Two separate problems compound this: 1. .github/workflows/test-modelmaker.yml's push/pull_request path filters only watched 'tinyml-modelmaker/**' -- a PR touching only tinyml-tinyverse, tinyml-modelzoo, or tinyml-modeloptimization (which is most of this session's PRs) never triggered the workflow to run at all, regardless of what test steps existed. 2. Even when the workflow does run, it only ever executes tests under tinyml-modelmaker/tests/ (four specific files via named Tier 1/2/3 steps). tinyml-tinyverse, tinyml-modelzoo, and torchmodelopt have no test-running step whatsoever -- any regression test added there is silently never checked on push or PR. Fixed by: - Restoring the tinyml-tinyverse/**, tinyml-modelzoo/**, and tinyml-modeloptimization/** path filters alongside tinyml-modelmaker/**, so the workflow actually triggers for PRs touching those packages. - Adding a test step for each of the three previously-uncovered packages. Each is guarded by a find-based existence check (not a plain `test -d tests`) because none of these tests/ directories exist yet on upstream/main -- an empty or missing directory makes pytest exit non-zero ("no tests collected"), which would otherwise fail CI on this package until the first PR adding tests to it merges. The guard lets this land safely now and start actually running tests the moment any PR (this session's or otherwise) adds them. Deliberately NOT adding a broader "run everything under tinyml-modelmaker/tests/" sweep beyond the existing four named Tier files, despite tinyml-modelmaker itself having several test files (e.g. test_constants.py, test_dataset_utils.py, test_nas_support.py) not wired into any Tier and therefore also never run: a full sweep surfaces 17 pre-existing failures unrelated to anything in this session (verified even the current, unmodified Tier 1 step alone already fails 2/447 tests locally), which would immediately turn this workflow red on merge. That gap is real but is its own, separate, pre-existing issue -- out of scope for a CI-triggering/coverage fix that should land safely. Verified locally: the find-based guard correctly skips tinyml-tinyverse, tinyml-modelzoo, and torchmodelopt on this branch (none of their tests/ directories have any content yet on upstream/main) and would correctly run pytest once a PR adds real test files there. Co-Authored-By: Claude Sonnet 5 --- .github/workflows/test-modelmaker.yml | 41 +++++++++++++++++++++++++++ 1 file changed, 41 insertions(+) diff --git a/.github/workflows/test-modelmaker.yml b/.github/workflows/test-modelmaker.yml index 7233d670..99c1d1e7 100644 --- a/.github/workflows/test-modelmaker.yml +++ b/.github/workflows/test-modelmaker.yml @@ -5,11 +5,17 @@ on: branches: [platypus_dev_1.3, main] paths: - 'tinyml-modelmaker/**' + - 'tinyml-tinyverse/**' + - 'tinyml-modelzoo/**' + - 'tinyml-modeloptimization/**' - '.github/workflows/test-modelmaker.yml' pull_request: branches: [platypus_dev_1.3, main] paths: - 'tinyml-modelmaker/**' + - 'tinyml-tinyverse/**' + - 'tinyml-modelzoo/**' + - 'tinyml-modeloptimization/**' workflow_dispatch: # manual trigger jobs: @@ -73,3 +79,38 @@ jobs: - name: Tier 2 — Pipeline Smoke Tests working-directory: tinyml-modelmaker run: python -m pytest tests/test_pipeline_smoke.py -v --tb=short + + # tinyml-tinyverse, tinyml-modelzoo, and torchmodelopt have no + # equivalent of the Tier steps above -- until now, nothing ran their + # tests/ directories in CI at all, so every regression test added + # there was silently never checked on push/PR. Guarded with a + # find-based existence check (rather than a plain `test -d tests`) + # since an empty tests/ directory makes pytest exit non-zero with "no + # tests collected", which would otherwise fail CI on packages that + # temporarily have no tests. + - name: tinyml-tinyverse Tests + working-directory: tinyml-tinyverse + run: | + if [ -d tests ] && find tests -name 'test_*.py' -print -quit | grep -q .; then + python -m pytest tests/ -v --tb=short + else + echo "No tests found in tinyml-tinyverse/tests/ yet -- skipping." + fi + + - name: tinyml-modelzoo Tests + working-directory: tinyml-modelzoo + run: | + if [ -d tests ] && find tests -name 'test_*.py' -print -quit | grep -q .; then + python -m pytest tests/ -v --tb=short + else + echo "No tests found in tinyml-modelzoo/tests/ yet -- skipping." + fi + + - name: torchmodelopt — Component Tests + working-directory: tinyml-modeloptimization/torchmodelopt + run: | + if [ -d tests ] && find tests -name 'test_*.py' -print -quit | grep -q .; then + python -m pytest tests/ -v --tb=short + else + echo "No tests found in tinyml-modeloptimization/torchmodelopt/tests/ yet -- skipping." + fi From 9775450e0236650ae0d59ae4275744f4951f0e8d Mon Sep 17 00:00:00 2001 From: M Platypus Date: Tue, 4 Aug 2026 20:35:34 -0400 Subject: [PATCH 2/3] fix: workflow-only PRs never triggered the CI they modify Independent Fable peer review of this PR caught an in-scope gap: the pull_request path filters omitted '.github/workflows/test-modelmaker.yml' itself (the push filters have had it all along -- an inherited asymmetry, not one this PR introduced). Consequence: a PR that only changes this workflow -- including this very PR -- never triggers the CI it modifies, so workflow changes merge unvalidated. Given this PR's whole purpose is "CI doesn't trigger when it should," that omission belongs in scope. One added path glob, mirroring the push filter. Co-Authored-By: Claude Sonnet 5 --- .github/workflows/test-modelmaker.yml | 3 +++ 1 file changed, 3 insertions(+) diff --git a/.github/workflows/test-modelmaker.yml b/.github/workflows/test-modelmaker.yml index 99c1d1e7..fb67c290 100644 --- a/.github/workflows/test-modelmaker.yml +++ b/.github/workflows/test-modelmaker.yml @@ -16,6 +16,9 @@ on: - 'tinyml-tinyverse/**' - 'tinyml-modelzoo/**' - 'tinyml-modeloptimization/**' + # Without this, a PR that only changes this workflow (like the one + # introducing this line) never triggers the very CI it modifies. + - '.github/workflows/test-modelmaker.yml' workflow_dispatch: # manual trigger jobs: From 550d0d8ad65b64b8e1b6f29c61d80dcfb993380b Mon Sep 17 00:00:00 2001 From: M Platypus Date: Tue, 4 Aug 2026 21:44:51 -0400 Subject: [PATCH 3/3] fix: test-presence guard missed pytest's *_test.py discovery pattern Both CodeRabbit and Fable reviews flagged that the find-based guard only matched test_*.py, so a package whose only tests use pytest's other default discovery pattern (foo_test.py) would be silently skipped in CI. Extended all three guards to match both patterns; verified the guard runs for a lone foo_test.py and still skips an empty tests/ directory. Co-Authored-By: Claude Sonnet 5 --- .github/workflows/test-modelmaker.yml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/.github/workflows/test-modelmaker.yml b/.github/workflows/test-modelmaker.yml index fb67c290..3901f409 100644 --- a/.github/workflows/test-modelmaker.yml +++ b/.github/workflows/test-modelmaker.yml @@ -94,7 +94,7 @@ jobs: - name: tinyml-tinyverse Tests working-directory: tinyml-tinyverse run: | - if [ -d tests ] && find tests -name 'test_*.py' -print -quit | grep -q .; then + if [ -d tests ] && find tests \( -name 'test_*.py' -o -name '*_test.py' \) -print -quit | grep -q .; then python -m pytest tests/ -v --tb=short else echo "No tests found in tinyml-tinyverse/tests/ yet -- skipping." @@ -103,7 +103,7 @@ jobs: - name: tinyml-modelzoo Tests working-directory: tinyml-modelzoo run: | - if [ -d tests ] && find tests -name 'test_*.py' -print -quit | grep -q .; then + if [ -d tests ] && find tests \( -name 'test_*.py' -o -name '*_test.py' \) -print -quit | grep -q .; then python -m pytest tests/ -v --tb=short else echo "No tests found in tinyml-modelzoo/tests/ yet -- skipping." @@ -112,7 +112,7 @@ jobs: - name: torchmodelopt — Component Tests working-directory: tinyml-modeloptimization/torchmodelopt run: | - if [ -d tests ] && find tests -name 'test_*.py' -print -quit | grep -q .; then + if [ -d tests ] && find tests \( -name 'test_*.py' -o -name '*_test.py' \) -print -quit | grep -q .; then python -m pytest tests/ -v --tb=short else echo "No tests found in tinyml-modeloptimization/torchmodelopt/tests/ yet -- skipping."