fix: pin dependency versions and Python support range - #74
Open
maan-iitd2 wants to merge 1 commit into
Open
Conversation
- great_expectations<1.0: ingen uses the legacy Dataset API
(ge.from_pandas().expect_*) that was removed in GE 1.0.
- pandas<3.0: pandas 3.0 flips Copy-on-Write on by default and drops
deprecated APIs that ingen's formatters/readers rely on.
- numpy left unbounded on the low end but effectively capped below 2
by great_expectations' own dependency; documented why in setup.py's
python_requires so a 3.13+ install fails with a clear message
instead of trying to build numpy from source.
- add python_requires='>=3.9,<3.13' so pip rejects incompatible
interpreters up front.
- add openpyxl as an explicit dependency: ingen's own file_reader and
writer call pandas' Excel read/write with engine='openpyxl', which
pandas does not install by default.
- setup.py: parse requirements.txt properly (skip blank/comment
lines) so the new inline comments above don't get fed to
install_requires as garbage requirements; fall back to a valid dev
version when the CI-only {{VERSION_PLACEHOLDER}} token hasn't been
substituted, since an unresolved token isn't a valid PEP 440 version
and breaks `pip install -e .` for every local/editable install.
Verified with a from-scratch venv (Python 3.12): `pip install -e .`
succeeds, and the full test suite goes from 393 passed/33 failed (as
on unmodified main) to 404 passed/22 failed -- the version pins fix
11 pre-existing failures (formatter and merger tests) caused by
dependency drift. Remaining 22 failures are pre-existing environment
issues unrelated to packaging (missing GPG binary, Windows file locks,
live network calls) and are unchanged in count/identity by this diff.
Also ran `python -m ingen` end-to-end against the fresh install.
Signed-off-by: maan-iitd2 <maan.iitd.ac.in@gmail.com>
Jatin-8898
approved these changes
Aug 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Pin dependency versions and Python support range in
setup.py/requirements.txt. No code changes.great_expectations>=0.18.9,<1.0ge.from_pandas(...).expect_*Dataset API, removed in GE 1.0. Unpinned,pip installcan resolve a GE version that silently breaks every validation.pandas>=2.2.2,<3.0openpyxl>=3.1.0(new)ingen/reader/file_reader.pyandingen/writer/writer.pycallpd.read_excel/to_excel(engine='openpyxl')directly, but pandas does not installopenpyxlby default — this was a missing, implicit dependency.setup.py: python_requires='>=3.9,<3.13'numpy<2(forced transitively by GE's legacy-API requirement) has no wheels for Python 3.13+, so an unconstrained install on 3.13+ falls into a from-source numpy build that needs a C compiler and fails with a confusing error. This makes pip reject it up front with a clear message.setup.py: read_requirements()open('requirements.txt').read().splitlines()fed every line — including the new comments above — straight intoinstall_requires. Replaced with a small filter that skips blank lines and#comments.setup.py: resolve_version(){{VERSION_PLACEHOLDER}}, substituted by CI (.github/workflows/python-publish.yml) viasedat release time. Locally, the token is left as-is, which isn't a valid PEP 440 version and makespip install -e .fail on a fresh checkout. Falls back to0.0.0.dev0when the token hasn't been substituted; the release path is untouched.Testing
Built a from-scratch venv (Python 3.12, no pre-existing installs):
pip install -e .onmainfails outright — the unresolved{{VERSION_PLACEHOLDER}}isn't a valid version string, sosetuptoolsraisesInvalidVersionduring dependency resolution.pip install -e .succeeds cleanly.main→ 404 passed / 22 failed with this change — the version pins fix 11 pre-existing failures (test_common_formatters.py,test_merger.py) caused by dependency drift on unpinned installs. The remaining 22 failures are pre-existing environment issues unrelated to packaging (missing GPG binary, Windows file locks, live network calls intest_api_reader/test_aiohttp_retry) — same failures, same count, on both branches.python -m ingenend-to-end against the fresh install (XML source → CSV output) to confirm the installed package actually runs.python_requiresdoes its job: installing into a Python 3.14 venv fails immediately withPackage 'ingen-lib' requires a different Python: 3.14.0 not in '<3.13,>=3.9'instead of a numpy build failure.