Phase 37: make the built wheel actually work - #24
Merged
Conversation
- git mv the three alert wavs from sounds/ to core/sounds/ so they can be package data
- git mv generate_alert_sounds.py to scripts/, out of the shipped runtime tree
- utils.SOUNDS_DIR now resolves via importlib.resources.files("core.sounds"), no fallback
- add tests/test_packaging.py asserting SOUNDS_DIR exists, is core/sounds, holds all 3 cues
- update README.md and CLAUDE.md sound paths
- declare [tool.setuptools.package-data] for core.sounds (*.wav, *.mp3) - declare web static/*, static/vendor/*, templates/* (vendor needs its own glob) - set include-package-data = true alongside py-modules - extend tests/test_packaging.py with the six web data files under the web package Verified against a real build: wheel grew 61 -> 71 entries, carrying 3 core/sounds wavs, 5 web/static files, 1 web/templates file, 0 top-level sounds/, 0 generator.
- 37-01-SUMMARY.md records the observed wheel entry counts (61 -> 71) - PKG-01 marked complete in REQUIREMENTS.md - ROADMAP.md phase 37 progress 1/4, In Progress - STATE.md position, decisions, metrics, session continuity
- [project] dependencies goes from 1 entry to 9: colorama, cryptography, keyring, nodriver, platformdirs, pydantic, pydantic-settings[yaml], pyyaml, requests. Every one is imported unconditionally in the production tree; pins match requirements.txt byte for byte. - colorama and pyyaml are logger.py imports that arrive on Windows only by accident (click win32 marker, uvicorn[standard]) and only via the web extra, so a bare install fails on any OS without them. - web extra gains starlette>=0.40 and websockets>=10.4 as floors, not pins, so they do not fight fastapi's and uvicorn's own resolution. - New sound extra holds pygame; utils.py already degrades gracefully and a headless install should not pull an audio stack. - New test extra holds pytest, pytest-asyncio, httpx; nothing on a runtime path imports httpx. - requirements.txt drops the dead selenium and webdriver-manager pins; the codebase uses nodriver. - version = 2.0.0 and all packaging tables left untouched.
- 37-02-SUMMARY.md records the before/after clean-install import matrix (3 of 8 -> 8 of 8) and the shoppybot --help exit-0 proof - PKG-02, PKG-03, PKG-04 marked complete - STATE.md position advanced to plan 3 of 4
- core/paths.py: bundled_plugins_dir() computed from __file__ directly, never via _repo_root(), so the monkeypatchable _REPO_ROOT_OVERRIDE cannot redirect a directory whose every .py file is exec_module'd (T-37-10) - core/orchestrator.py:814 and core/service.py:133,159 call the accessor instead of inlining Path(__file__).parent.parent / "plugins" - drop the now-unused pathlib.Path import from both callers - tests: behavior preservation against core.orchestrator.__file__ and core.service.__file__, override independence, 7-plugin contents, and a seam guard so the expression cannot be re-inlined (Phase 43 criterion 5) - core/registry.py unchanged: it already takes plugins_dir as a parameter
…ch flag - Phase 43's research flag stated PKG-06 as an open question that could push importlib.resources work back into Phase 37; it now states the answer - bundled_plugins_dir() verified from a clean-venv wheel install: resolves to site-packages/plugins with all 7 shopbot_plugin_*.py files present - retains the scout's namespace caveat, the bundled root lands as a top-level site-packages/plugins entry that another distribution could collide with - EXT-03 unblocked; no other phase entry touched
- 37-03-SUMMARY.md records the PKG-06 answer as the durable input Phase 43 consumes - STATE.md: position to plan 4 of 4, 3 decisions, 37-03 metric row, 37-04 carry note - ROADMAP.md: Phase 37 plan progress 3/4, 37-03 checkbox - REQUIREMENTS.md: PKG-06 marked complete
Encodes the five locked wheel-health assertions in a stdlib-only script so the facts Phase 37 established cannot silently regress on the next dependency bump. - installs the one wheel in --wheel-dir into a fresh venv by absolute path, never from a pinned dev requirements file and never from source - runs the assertions in cheapest-failing-first order, stopping at the first failure with the offending subprocess output, so CI names the broken layer - assertion 3 checks both halves of PKG-05: the wheel ships entries under web/static, web/templates and core/sounds, AND notification.wav resolves from utils.SOUNDS_DIR inside the clean install - assertion 5 polls real HTTP, because core/cli/web.py prints the dashboard URL before create_app() runs, so the printed line proves nothing - wheel is resolved by glob, never by a versioned filename, so release-please bumping the version cannot break the job
Runs on ubuntu-latest and windows-latest with fail-fast disabled, builds the wheel, and hands it to scripts/verify_wheel.py, which creates its own clean virtualenv and installs the wheel there by absolute path. - separate install step from the test job on purpose: the test job installs requirements.txt first, deliberately, and doing the same here would hide every dependency and data file missing from the wheel itself - ubuntu is not optional; colorama and pyyaml reach a Windows install transitively, so a Windows-only job would keep missing that failure mode - SHOPBOT_DATA_DIR and WHEEL_VENV stay at step level: the runner context does not exist in job-level env and using it there fails the whole workflow to compile rather than failing one job - actions kept on @v6 tags; Phase 38 owns SHA pinning and the permissions block test job byte-identical to HEAD; diff is 37 additions and 0 deletions.
PKG-05 closed, and Phase 37 with it. All six PKG requirements are now proven against a real built wheel rather than against pyproject.toml. - 37-04-SUMMARY.md records the five PASS lines verbatim and both negative controls verbatim, with exit codes - STATE.md: Phase 37 marked complete, 13 of 84 requirements, next is Phase 38 - ROADMAP.md phase-status row 37 to 4/4 Complete - REQUIREMENTS.md PKG-05 checked off and its traceability row set to Complete The wheel CI job has NOT run on a runner. It lands with this branch's push.
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Comment on lines
+56
to
+82
| runs-on: ${{ matrix.os }} | ||
| strategy: | ||
| fail-fast: false | ||
| matrix: | ||
| os: [ubuntu-latest, windows-latest] | ||
|
|
||
| steps: | ||
| - uses: actions/checkout@v6 | ||
| - uses: actions/setup-python@v6 | ||
| with: | ||
| python-version: "3.13" | ||
| - name: Install build tooling | ||
| # Only pip and build. Nothing here reaches the clean venv, which the | ||
| # verification script creates itself and installs the wheel into. | ||
| run: | | ||
| python -m pip install --upgrade pip | ||
| pip install build | ||
| - name: Build wheel | ||
| run: python -m build --wheel --outdir dist | ||
| - name: Verify wheel | ||
| env: | ||
| # Same step-level rule as the Test step above: the `runner` context does | ||
| # not exist in job-level `env:`, and using it there fails the whole | ||
| # workflow to compile rather than failing this one job. | ||
| SHOPBOT_DATA_DIR: ${{ runner.temp }}/shopbot-wheel | ||
| WHEEL_VENV: ${{ runner.temp }}/wheelenv | ||
| run: python scripts/verify_wheel.py --wheel-dir dist --venv "${{ env.WHEEL_VENV }}" |
…e-close # Conflicts: # .planning/REQUIREMENTS.md # .planning/ROADMAP.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the v5.0 Phase 37 requirements PKG-01 through PKG-06.
The problem
The built wheel was not degraded, it was dead.
shoppybot --helpfailed at import onpydantic_settingsbefore reaching any command dispatch. Measured, not inferred.What changed
shoppybot --helpfrom clean installModuleNotFoundErrorsounds/moved intocore/sounds/as a real package. It was structurally unshippable before:utilsis a top-level module, soos.path.dirname(__file__)resolved tosite-packages/sounds, which cannot be package data of anything. Pluspackage-dataglobs forweb/staticandweb/templates.coloramaandpyyamlwere reaching Windows installs only by accident viaclick's win32 marker anduvicorn[standard], and only under thewebextra, so a barepip install shoppybotfailed on every OS.seleniumandwebdriver-managerpins removed.scripts/verify_wheel.py, a five-assertion gate, plus a CIwheeljob on ubuntu and windows that installs withoutrequirements.txt.bundled_plugins_dir()added as the named plugin-root seam. Computed from__file__, deliberately not via_repo_root(), which honours a test-monkeypatched override. Three inline call sites collapsed into it.Phase 43 gate: answered, favorably
PKG-06 was a declared hard gate on Phase 43. The bundled plugin root does survive a wheel install:
site-packages/pluginswith all 7shopbot_plugin_*.py. Noimportlib.resourcesrewrite needed. Phase 43 is unblocked.Verification
Local suite 969 passed / 2 skipped. Every packaging claim was checked by building a real wheel and probing it in a clean venv, never by reading
pyproject.tomlback.The wheel gate's negative control was run for real, twice, because assertion 3 makes two independent claims. Doctored wheels produced exit 1 naming assertion 3: once for files missing from the wheel, once for files shipping but not resolving at runtime. That second case is exactly what a zip-entry-only check would have reported green.
version = "2.0.0"untouched; release-please owns versioning.Note
This PR is also what first exercises the new
wheelCI job. The ubuntu leg has never run.