From 110d6847b9b5e2e362d0ee4194feafccb34487d3 Mon Sep 17 00:00:00 2001 From: Grzegorz Bokota Date: Mon, 27 Jul 2026 09:36:35 +0200 Subject: [PATCH 1/9] Improve tox usage documentation --- docs/developers/contributing/testing.md | 51 ++++++++++++++++++++++++- 1 file changed, 49 insertions(+), 2 deletions(-) diff --git a/docs/developers/contributing/testing.md b/docs/developers/contributing/testing.md index dada85d8e..b4323f2b5 100644 --- a/docs/developers/contributing/testing.md +++ b/docs/developers/contributing/testing.md @@ -143,18 +143,65 @@ in `System Settings > Privacy & Security > Accessibility` so `pyautogui` can con It is also possible to run tests locally using `tox`. We use `tox` to run test in CI. The main difference between running `pytest` locally or `tox` locally is that `tox` will create a virtual environment for each test environment, so it will take a bit more time. Though, `tox` will be more similar to the CI environment. +The only requirement for running tests is to have `tox` in your environment and the target Python version discoverable on your system. To run test using `tox` using Python 3.13 and pyqt6 on Linux, enter: ```sh -tox -e py313-linux-pyqt6 +tox -e py313-linux-pyqt6-no_cov ``` -To get list of all available environments that may be run: +To get the list of all available environments that may be run: ```sh tox list ``` +#### Speed up tox environment creation + +To speed up the setup of creation of tox environments you can install `tox-uv` plugin that uses `uv` in +place of `pip` plus `virtualenv` to create tox environments. +The only disadvantage of this method is that it install `uv` in your environment +that is making precedence over global `uv` installation. +So you need to remember to regularly update one more `uv` installation. + +#### Running with constraints + +For fully reproduce CI environment you might use constraints files for tox environment creation. +For example, to run tests using Python 3.13 and pyqt6 on Linux with a constraints file, enter: + +```sh +PIP_CONSTRAINT=resources/constraints/constraints_py313.txt tox -e py313-linux-pyqt6-no_cov +``` + +or with `tox-uv` + +```sh +UV_CONSTRAINT=resources/constraints/constraints_py313.txt tox -e py313-linux-pyqt6-no_cov +``` + +#### Running a subset of test using tox + +The part of tox call after `--` is passed to pytest. So you can run a subset of tests using tox as well. For example, to run only tests in `src/napari/layers/image/` file using Python 3.13 and pyqt6 on Linux, enter: + +```sh +tox -e py313-linux-pyqt6-no_cov -- src/napari/layers/image +``` + +#### Use tox to create an environment for debugging + +The `tox` provides a convenient way to create a virtual environment for debugging. You can use `devenv` command to create a virtual environment for debugging. For example, to create a virtual environment for debugging using Python 3.13 and pyqt6 on Linux, enter: + +```sh +$ tox devenv -e py313-linux-pyqt6-no_cov +``` +Then at the end of the output is a path to created virtual environment. + +```sh +... +ROOT: created development environment under /Users/grzegorzbokota/Documents/Projekty/napari/venv +``` +That you can activate in your IDE or terminal as a normal virtual environment. + ### Run tests without pop-up windows Some tests create visible napari viewers, which pop up on your monitor then quickly disappear. From 69029efa4e49348ba61518c37269785589db14f2 Mon Sep 17 00:00:00 2001 From: Grzegorz Bokota Date: Mon, 27 Jul 2026 09:53:35 +0200 Subject: [PATCH 2/9] add infirmation about minimum requirement testing --- docs/developers/contributing/testing.md | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/docs/developers/contributing/testing.md b/docs/developers/contributing/testing.md index b4323f2b5..ff168fd08 100644 --- a/docs/developers/contributing/testing.md +++ b/docs/developers/contributing/testing.md @@ -156,6 +156,26 @@ To get the list of all available environments that may be run: tox list ``` +#### Minimum requirements testing. + +One of our test suite entries is testing napari against minimum requirements declared in `pyproject.toml` file. +This checks if napari will still work, even if someone have outdated dependencies installed in their environment. +Most often if something is failing in this test run it is either a simple change or reason to bump minimum requirements in `pyproject.toml` file. + +To run this test use: + +```sh +MIN_REQ=1 tox -e py311-linux-pyqt5-no_cov --recreate +``` + +or + +```sh +MIN_REQ=1 tox -e py311-windows-pyqt5-no_cov --recreate +``` + +Unfortunately, it is impossible to test this on arm macOS, because of lack of pyqt5 support for this platform. + #### Speed up tox environment creation To speed up the setup of creation of tox environments you can install `tox-uv` plugin that uses `uv` in @@ -179,6 +199,8 @@ or with `tox-uv` UV_CONSTRAINT=resources/constraints/constraints_py313.txt tox -e py313-linux-pyqt6-no_cov ``` +Constraint usage guarantees the same version of dependencies as in CI environment. Its usually isn’t needed, but it sometimes happens that the newest version of some dependency is not compatible with napari test suite, and it is good to have a way to run tests with the same versions of dependencies as in the CI environment. + #### Running a subset of test using tox The part of tox call after `--` is passed to pytest. So you can run a subset of tests using tox as well. For example, to run only tests in `src/napari/layers/image/` file using Python 3.13 and pyqt6 on Linux, enter: From 2e08e87dce3907256c91298539a14e642f816dba Mon Sep 17 00:00:00 2001 From: Grzegorz Bokota Date: Thu, 6 Aug 2026 15:13:51 +0200 Subject: [PATCH 3/9] simplify commands, add information about coverage --- docs/developers/contributing/testing.md | 64 ++++++++++++++----------- 1 file changed, 35 insertions(+), 29 deletions(-) diff --git a/docs/developers/contributing/testing.md b/docs/developers/contributing/testing.md index ff168fd08..897ede11e 100644 --- a/docs/developers/contributing/testing.md +++ b/docs/developers/contributing/testing.md @@ -144,10 +144,10 @@ It is also possible to run tests locally using `tox`. We use `tox` to run test i The main difference between running `pytest` locally or `tox` locally is that `tox` will create a virtual environment for each test environment, so it will take a bit more time. Though, `tox` will be more similar to the CI environment. The only requirement for running tests is to have `tox` in your environment and the target Python version discoverable on your system. -To run test using `tox` using Python 3.13 and pyqt6 on Linux, enter: +To run test using `tox` using Python 3.13 and pyqt6, enter: ```sh -tox -e py313-linux-pyqt6-no_cov +tox -e py313-pyqt6 ``` To get the list of all available environments that may be run: @@ -165,38 +165,18 @@ Most often if something is failing in this test run it is either a simple change To run this test use: ```sh -MIN_REQ=1 tox -e py311-linux-pyqt5-no_cov --recreate +MIN_REQ=1 tox -e py311-pyqt5 --recreate ``` -or - -```sh -MIN_REQ=1 tox -e py311-windows-pyqt5-no_cov --recreate -``` - -Unfortunately, it is impossible to test this on arm macOS, because of lack of pyqt5 support for this platform. - -#### Speed up tox environment creation - -To speed up the setup of creation of tox environments you can install `tox-uv` plugin that uses `uv` in -place of `pip` plus `virtualenv` to create tox environments. -The only disadvantage of this method is that it install `uv` in your environment -that is making precedence over global `uv` installation. -So you need to remember to regularly update one more `uv` installation. +Unfortunately, it is impossible to test this on ARM macOS, because of lack of pyqt5 support for this platform. #### Running with constraints For fully reproduce CI environment you might use constraints files for tox environment creation. -For example, to run tests using Python 3.13 and pyqt6 on Linux with a constraints file, enter: +For example, to run tests using Python 3.13 and pyqt6 with a constraints file, enter: ```sh -PIP_CONSTRAINT=resources/constraints/constraints_py313.txt tox -e py313-linux-pyqt6-no_cov -``` - -or with `tox-uv` - -```sh -UV_CONSTRAINT=resources/constraints/constraints_py313.txt tox -e py313-linux-pyqt6-no_cov +UV_CONSTRAINT=resources/constraints/constraints_py313.txt tox -e py313-pyqt6 ``` Constraint usage guarantees the same version of dependencies as in CI environment. Its usually isn’t needed, but it sometimes happens that the newest version of some dependency is not compatible with napari test suite, and it is good to have a way to run tests with the same versions of dependencies as in the CI environment. @@ -206,15 +186,17 @@ Constraint usage guarantees the same version of dependencies as in CI environmen The part of tox call after `--` is passed to pytest. So you can run a subset of tests using tox as well. For example, to run only tests in `src/napari/layers/image/` file using Python 3.13 and pyqt6 on Linux, enter: ```sh -tox -e py313-linux-pyqt6-no_cov -- src/napari/layers/image +tox -e py313-pyqt6 -- src/napari/layers/image ``` #### Use tox to create an environment for debugging -The `tox` provides a convenient way to create a virtual environment for debugging. You can use `devenv` command to create a virtual environment for debugging. For example, to create a virtual environment for debugging using Python 3.13 and pyqt6 on Linux, enter: +The `tox` provides a convenient way to create a virtual environment for debugging. +You can use `devenv` command to create a virtual environment for debugging. +For example, to create a virtual environment for debugging using Python 3.13 and pyqt6, enter: ```sh -$ tox devenv -e py313-linux-pyqt6-no_cov +$ tox devenv -e py313-pyqt6 ``` Then at the end of the output is a path to created virtual environment. @@ -224,6 +206,30 @@ ROOT: created development environment under /Users/grzegorzbokota/Documents/Proj ``` That you can activate in your IDE or terminal as a normal virtual environment. +#### Run test measuring coverage using tox + +You might want to locally check how your changes affect test coverage. You can run tox with coverage measurement using the following command: + +```sh +TOX_TEST_RUNNER="coverage run" tox -e py314-pyqt6 +``` + +This command will create a coverage report in `.coverage` file. You can then generate a report using the following command: + +1. `coverage report` - to see a report in the console. It will show you the percentage of code covered by tests and the lines that are not covered. +2. `coverage html` - to generate a report in HTML format. You can then open it browser and see exactly which lines are not covered by tests. + +```{note} +We do not use `pytes-cov` as it do not measure coverage of all code, like our `make-napari-viewer` fixture. +``` + +```{note} +Some parts of code are tested only on given platform, python version or in minimum requirements test. +So running coverage locally might not be the same as in CI. +Some lines might be not covered on your platform, but they are covered in CI. +``` + + ### Run tests without pop-up windows Some tests create visible napari viewers, which pop up on your monitor then quickly disappear. From e08172ad417c915ae341a95bdd324ff93aefa31d Mon Sep 17 00:00:00 2001 From: Grzegorz Bokota Date: Thu, 6 Aug 2026 15:25:21 +0200 Subject: [PATCH 4/9] remove leftover system information --- docs/developers/contributing/testing.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/developers/contributing/testing.md b/docs/developers/contributing/testing.md index 897ede11e..999a0df87 100644 --- a/docs/developers/contributing/testing.md +++ b/docs/developers/contributing/testing.md @@ -183,7 +183,8 @@ Constraint usage guarantees the same version of dependencies as in CI environmen #### Running a subset of test using tox -The part of tox call after `--` is passed to pytest. So you can run a subset of tests using tox as well. For example, to run only tests in `src/napari/layers/image/` file using Python 3.13 and pyqt6 on Linux, enter: +The part of tox call after `--` is passed to pytest. So you can run a subset of tests using tox as well. +For example, to run only tests in `src/napari/layers/image/` file using Python 3.13 and pyqt6, enter: ```sh tox -e py313-pyqt6 -- src/napari/layers/image From b507b9c30f691a7f8e424151fca4ba6d290db7ed Mon Sep 17 00:00:00 2001 From: Grzegorz Bokota Date: Fri, 7 Aug 2026 15:20:38 +0200 Subject: [PATCH 5/9] Apply suggestions from code review Co-authored-by: Lorenzo Gaifas --- docs/developers/contributing/testing.md | 29 ++++++++++--------------- 1 file changed, 12 insertions(+), 17 deletions(-) diff --git a/docs/developers/contributing/testing.md b/docs/developers/contributing/testing.md index 999a0df87..8ae7e74ea 100644 --- a/docs/developers/contributing/testing.md +++ b/docs/developers/contributing/testing.md @@ -158,9 +158,8 @@ tox list #### Minimum requirements testing. -One of our test suite entries is testing napari against minimum requirements declared in `pyproject.toml` file. -This checks if napari will still work, even if someone have outdated dependencies installed in their environment. -Most often if something is failing in this test run it is either a simple change or reason to bump minimum requirements in `pyproject.toml` file. +One of our test suite entries is testing napari against the minimum requirements declared in the `pyproject.toml` file. +This checks if napari will still work even with outdated dependencies. Most often if something is failing in this test run, it is either a simple change or a reason to bump the minimum requirements in the `pyproject.toml` file. To run this test use: @@ -168,23 +167,22 @@ To run this test use: MIN_REQ=1 tox -e py311-pyqt5 --recreate ``` -Unfortunately, it is impossible to test this on ARM macOS, because of lack of pyqt5 support for this platform. +Unfortunately, it is impossible to test this on ARM macOS, due to the lack of pyqt5 support for this platform. #### Running with constraints -For fully reproduce CI environment you might use constraints files for tox environment creation. -For example, to run tests using Python 3.13 and pyqt6 with a constraints file, enter: +To fully reproduce the CI environment, you might use our constraints files; for example, to run tests using Python 3.13 and pyqt6 with a constraints file, enter: ```sh UV_CONSTRAINT=resources/constraints/constraints_py313.txt tox -e py313-pyqt6 ``` -Constraint usage guarantees the same version of dependencies as in CI environment. Its usually isn’t needed, but it sometimes happens that the newest version of some dependency is not compatible with napari test suite, and it is good to have a way to run tests with the same versions of dependencies as in the CI environment. +Constraint usage guarantees the same version of dependencies as in the CI environment. It usually isn’t needed, but it sometimes happens that the newest version of some dependency is not compatible with the napari test suite, and it is useful to have a way to run tests with the same versions of dependencies as in the CI environment. #### Running a subset of test using tox The part of tox call after `--` is passed to pytest. So you can run a subset of tests using tox as well. -For example, to run only tests in `src/napari/layers/image/` file using Python 3.13 and pyqt6, enter: +For example, to run only tests in the `src/napari/layers/image/` file using Python 3.13 and pyqt6, enter: ```sh tox -e py313-pyqt6 -- src/napari/layers/image @@ -192,8 +190,7 @@ tox -e py313-pyqt6 -- src/napari/layers/image #### Use tox to create an environment for debugging -The `tox` provides a convenient way to create a virtual environment for debugging. -You can use `devenv` command to create a virtual environment for debugging. +`tox` provides a convenient way to create a virtual environment for debugging via the `devenv` command. For example, to create a virtual environment for debugging using Python 3.13 and pyqt6, enter: ```sh @@ -203,9 +200,9 @@ Then at the end of the output is a path to created virtual environment. ```sh ... -ROOT: created development environment under /Users/grzegorzbokota/Documents/Projekty/napari/venv +ROOT: created development environment under [...]/napari/venv ``` -That you can activate in your IDE or terminal as a normal virtual environment. +You can activate it in your IDE or terminal as a normal virtual environment. #### Run test measuring coverage using tox @@ -218,16 +215,14 @@ TOX_TEST_RUNNER="coverage run" tox -e py314-pyqt6 This command will create a coverage report in `.coverage` file. You can then generate a report using the following command: 1. `coverage report` - to see a report in the console. It will show you the percentage of code covered by tests and the lines that are not covered. -2. `coverage html` - to generate a report in HTML format. You can then open it browser and see exactly which lines are not covered by tests. +2. `coverage html` - to generate a report in HTML format. You can then open it in the browser and see exactly which lines are not covered by tests. ```{note} -We do not use `pytes-cov` as it do not measure coverage of all code, like our `make-napari-viewer` fixture. +We do not use `pytest-cov` as it do not measure coverage of all code, like our `make-napari-viewer` fixture. ``` ```{note} -Some parts of code are tested only on given platform, python version or in minimum requirements test. -So running coverage locally might not be the same as in CI. -Some lines might be not covered on your platform, but they are covered in CI. +Some parts of the code are tested only on a given platform, python version, or in the minimum requirements test, so running coverage locally might not give the same results as on CI. ``` From e1d75792968471fe13a7a6cde7bc294edcc9333c Mon Sep 17 00:00:00 2001 From: Grzegorz Bokota Date: Mon, 10 Aug 2026 16:21:17 +0200 Subject: [PATCH 6/9] Apply suggestions from code review Co-authored-by: Carol Willing --- docs/developers/contributing/testing.md | 23 +++++++++++++---------- 1 file changed, 13 insertions(+), 10 deletions(-) diff --git a/docs/developers/contributing/testing.md b/docs/developers/contributing/testing.md index 8ae7e74ea..f2c464ebb 100644 --- a/docs/developers/contributing/testing.md +++ b/docs/developers/contributing/testing.md @@ -143,25 +143,27 @@ in `System Settings > Privacy & Security > Accessibility` so `pyautogui` can con It is also possible to run tests locally using `tox`. We use `tox` to run test in CI. The main difference between running `pytest` locally or `tox` locally is that `tox` will create a virtual environment for each test environment, so it will take a bit more time. Though, `tox` will be more similar to the CI environment. -The only requirement for running tests is to have `tox` in your environment and the target Python version discoverable on your system. -To run test using `tox` using Python 3.13 and pyqt6, enter: +The only requirement for running tests is to have `tox` in your environment and the target Python version discoverable on your system. + +`tox` can be used to run tests for a single environment configuration too. +For example, to run tests using `tox` for Python 3.13 and pyqt6, enter: ```sh tox -e py313-pyqt6 ``` -To get the list of all available environments that may be run: +To get the list of all available environment configurations that may be run: ```sh tox list ``` -#### Minimum requirements testing. +#### Minimum requirements testing -One of our test suite entries is testing napari against the minimum requirements declared in the `pyproject.toml` file. -This checks if napari will still work even with outdated dependencies. Most often if something is failing in this test run, it is either a simple change or a reason to bump the minimum requirements in the `pyproject.toml` file. +Tests may be run against the minimum requirements declared in the `pyproject.toml` file. +This checks if napari will work even with outdated or minimal dependencies. If the minimal requirements test run fails, it is likely either a simple regression fix or a reason to bump the minimum requirements in the `pyproject.toml` file. -To run this test use: +To run this test, prefix the `tox` command with `MIN_REQ=1`: ```sh MIN_REQ=1 tox -e py311-pyqt5 --recreate @@ -171,7 +173,8 @@ Unfortunately, it is impossible to test this on ARM macOS, due to the lack of py #### Running with constraints -To fully reproduce the CI environment, you might use our constraints files; for example, to run tests using Python 3.13 and pyqt6 with a constraints file, enter: +To fully reproduce the CI environment, you might use our constraints files which fully specifies dependency versions. +For example, to run tests using Python 3.13 and pyqt6 with a constraints file, enter: ```sh UV_CONSTRAINT=resources/constraints/constraints_py313.txt tox -e py313-pyqt6 @@ -181,7 +184,7 @@ Constraint usage guarantees the same version of dependencies as in the CI enviro #### Running a subset of test using tox -The part of tox call after `--` is passed to pytest. So you can run a subset of tests using tox as well. +To run a subset of tests using tox, add a directory or file after `--` and that will be passed to pytest. For example, to run only tests in the `src/napari/layers/image/` file using Python 3.13 and pyqt6, enter: ```sh @@ -204,7 +207,7 @@ ROOT: created development environment under [...]/napari/venv ``` You can activate it in your IDE or terminal as a normal virtual environment. -#### Run test measuring coverage using tox +#### Use tox to measure coverage You might want to locally check how your changes affect test coverage. You can run tox with coverage measurement using the following command: From 123fb96c434c76e91973f8cc66bb7bc368f809d8 Mon Sep 17 00:00:00 2001 From: Grzegorz Bokota Date: Mon, 10 Aug 2026 20:25:54 +0200 Subject: [PATCH 7/9] Remove information about discoverability of python --- docs/developers/contributing/testing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/developers/contributing/testing.md b/docs/developers/contributing/testing.md index f2c464ebb..946ed2d0c 100644 --- a/docs/developers/contributing/testing.md +++ b/docs/developers/contributing/testing.md @@ -143,7 +143,7 @@ in `System Settings > Privacy & Security > Accessibility` so `pyautogui` can con It is also possible to run tests locally using `tox`. We use `tox` to run test in CI. The main difference between running `pytest` locally or `tox` locally is that `tox` will create a virtual environment for each test environment, so it will take a bit more time. Though, `tox` will be more similar to the CI environment. -The only requirement for running tests is to have `tox` in your environment and the target Python version discoverable on your system. +The only requirement for running tests is to have `tox` in your environment. `tox` can be used to run tests for a single environment configuration too. For example, to run tests using `tox` for Python 3.13 and pyqt6, enter: From 5641672ef60f45ae05139c887c010cc56ec09cbd Mon Sep 17 00:00:00 2001 From: Grzegorz Bokota Date: Mon, 10 Aug 2026 21:11:37 +0200 Subject: [PATCH 8/9] Update docs/developers/contributing/testing.md Co-authored-by: Carol Willing --- docs/developers/contributing/testing.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/developers/contributing/testing.md b/docs/developers/contributing/testing.md index 946ed2d0c..f4ab2ee51 100644 --- a/docs/developers/contributing/testing.md +++ b/docs/developers/contributing/testing.md @@ -180,7 +180,9 @@ For example, to run tests using Python 3.13 and pyqt6 with a constraints file, e UV_CONSTRAINT=resources/constraints/constraints_py313.txt tox -e py313-pyqt6 ``` -Constraint usage guarantees the same version of dependencies as in the CI environment. It usually isn’t needed, but it sometimes happens that the newest version of some dependency is not compatible with the napari test suite, and it is useful to have a way to run tests with the same versions of dependencies as in the CI environment. +Constraint usage guarantees the same version of dependencies as in the CI environment where dependencies are pinned to a specific PyPI package version. + +While it usually isn’t needed to run this locally, running tests with the constraint file ensures that tests run with the exact same versions of dependencies as in napari's CI environment. #### Running a subset of test using tox From 34b59e78be87e062329e92869ad1f3ef85fdb1f3 Mon Sep 17 00:00:00 2001 From: Tim Monko Date: Mon, 10 Aug 2026 14:43:53 -0500 Subject: [PATCH 9/9] Apply suggestion from @TimMonko --- docs/developers/contributing/testing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/developers/contributing/testing.md b/docs/developers/contributing/testing.md index f4ab2ee51..4b60db6a2 100644 --- a/docs/developers/contributing/testing.md +++ b/docs/developers/contributing/testing.md @@ -223,7 +223,7 @@ This command will create a coverage report in `.coverage` file. You can then gen 2. `coverage html` - to generate a report in HTML format. You can then open it in the browser and see exactly which lines are not covered by tests. ```{note} -We do not use `pytest-cov` as it do not measure coverage of all code, like our `make-napari-viewer` fixture. +We do not use `pytest-cov` as it does not measure coverage of all code, like our `make-napari-viewer` fixture. ``` ```{note}