diff --git a/docs/developers/contributing/testing.md b/docs/developers/contributing/testing.md index dada85d8e..4b60db6a2 100644 --- a/docs/developers/contributing/testing.md +++ b/docs/developers/contributing/testing.md @@ -143,18 +143,94 @@ in `System Settings > Privacy & Security > Accessibility` so `pyautogui` can con It is also possible to run tests locally using `tox`. We use `tox` to run test in CI. The main difference between running `pytest` locally or `tox` locally is that `tox` will create a virtual environment for each test environment, so it will take a bit more time. Though, `tox` will be more similar to the CI environment. -To run test using `tox` using Python 3.13 and pyqt6 on Linux, enter: +The only requirement for running tests is to have `tox` in your environment. + +`tox` can be used to run tests for a single environment configuration too. +For example, to run tests using `tox` for Python 3.13 and pyqt6, enter: ```sh -tox -e py313-linux-pyqt6 +tox -e py313-pyqt6 ``` -To get list of all available environments that may be run: +To get the list of all available environment configurations that may be run: ```sh tox list ``` +#### Minimum requirements testing + +Tests may be run against the minimum requirements declared in the `pyproject.toml` file. +This checks if napari will work even with outdated or minimal dependencies. If the minimal requirements test run fails, it is likely either a simple regression fix or a reason to bump the minimum requirements in the `pyproject.toml` file. + +To run this test, prefix the `tox` command with `MIN_REQ=1`: + +```sh +MIN_REQ=1 tox -e py311-pyqt5 --recreate +``` + +Unfortunately, it is impossible to test this on ARM macOS, due to the lack of pyqt5 support for this platform. + +#### Running with constraints + +To fully reproduce the CI environment, you might use our constraints files which fully specifies dependency versions. +For example, to run tests using Python 3.13 and pyqt6 with a constraints file, enter: + +```sh +UV_CONSTRAINT=resources/constraints/constraints_py313.txt tox -e py313-pyqt6 +``` + +Constraint usage guarantees the same version of dependencies as in the CI environment where dependencies are pinned to a specific PyPI package version. + +While it usually isn’t needed to run this locally, running tests with the constraint file ensures that tests run with the exact same versions of dependencies as in napari's CI environment. + +#### Running a subset of test using tox + +To run a subset of tests using tox, add a directory or file after `--` and that will be passed to pytest. +For example, to run only tests in the `src/napari/layers/image/` file using Python 3.13 and pyqt6, enter: + +```sh +tox -e py313-pyqt6 -- src/napari/layers/image +``` + +#### Use tox to create an environment for debugging + +`tox` provides a convenient way to create a virtual environment for debugging via the `devenv` command. +For example, to create a virtual environment for debugging using Python 3.13 and pyqt6, enter: + +```sh +$ tox devenv -e py313-pyqt6 +``` +Then at the end of the output is a path to created virtual environment. + +```sh +... +ROOT: created development environment under [...]/napari/venv +``` +You can activate it in your IDE or terminal as a normal virtual environment. + +#### Use tox to measure coverage + +You might want to locally check how your changes affect test coverage. You can run tox with coverage measurement using the following command: + +```sh +TOX_TEST_RUNNER="coverage run" tox -e py314-pyqt6 +``` + +This command will create a coverage report in `.coverage` file. You can then generate a report using the following command: + +1. `coverage report` - to see a report in the console. It will show you the percentage of code covered by tests and the lines that are not covered. +2. `coverage html` - to generate a report in HTML format. You can then open it in the browser and see exactly which lines are not covered by tests. + +```{note} +We do not use `pytest-cov` as it does not measure coverage of all code, like our `make-napari-viewer` fixture. +``` + +```{note} +Some parts of the code are tested only on a given platform, python version, or in the minimum requirements test, so running coverage locally might not give the same results as on CI. +``` + + ### Run tests without pop-up windows Some tests create visible napari viewers, which pop up on your monitor then quickly disappear.