Fix/testsuitegate package import matching - #199
Conversation
TestSuiteGate.verify ran pytest via a hard-coded python3 interpreter with cwd set to file_path.parent. For nested files this misses the repo's pyproject.toml / pytest.ini / conftest.py and may use a different interpreter/venv than the host process, causing false positives or undiscovered tests and CI-vs-local mismatch. - Run pytest via sys.executable instead of python3. - Set cwd to the resolved project root (project_root or file dir). - Prepend the project root to PYTHONPATH for layouts that rely on it. - Add tests asserting the interpreter, cwd, and PYTHONPATH. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
_find_relevant_tests previously matched tests to the changed file using only file_path.stem, so a test doing `from mypkg.submodule import foo` was never attributed to mypkg/submodule/foo.py and the gate silently skipped real coverage. - _module_targets(): map the changed file to a set of qualified module names (project-root-relative and package-root-relative via the __init__.py chain), keeping the bare stem as a loose fallback - _imports_module(): match absolute imports by qualified name, including `from pkg.sub import leaf` where leaf is a submodule - _relative_import_hits(): resolve `from . import x` style imports on the filesystem and compare directly to the file under verification - lower confidence 0.9 -> 0.6 when no tests match, since an uncovered change is not the assurance a passing run gives; note clarifies it - cache test results per file path instead of per stem - new TestPackageImportMatching tests for package layouts Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
Warning Rate limit exceeded
You’ve run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
This pull request has been automatically marked as stale because it has not had recent activity for 14 days. If you believe this PR is still relevant, please:
This helps us keep the repository focused on active contributions. Thank you! 🙏 |
|
This pull request has been automatically closed because it was marked as stale and had no recent activity for 3 days. If you believe this PR should be reopened, please:
Thank you for your understanding! 🙏 |
1 participant
solve #192
This change fixes TestSuiteGate so it correctly attributes tests to the file under verification when packages are involved. Previously _find_relevant_tests identified the changed module using only file_path.stem, and _imports_module compared that bare stem against test import targets. As a result, a test importing from mypkg.submodule import foo was never matched to mypkg/submodule/foo.py, package-relative imports such as from . import x were not attributed at all, and when no tests matched the gate passed with high confidence — a potential false negative that let autofix ship changes whose tests existed in the repo but were never run. The fix introduces _module_targets, which maps the changed file to a set of qualified module names — both the project-root-relative dotted path and the package-root-relative path derived by walking up the init.py chain — while keeping the bare stem as a loose fallback for flat layouts. _imports_module now matches absolute imports by qualified name (including the from pkg.sub import leaf case where leaf is a submodule), and a new relative_import_hits helper resolves relative imports on the filesystem and compares them directly to the file under verification. When no tests match, confidence is lowered from 0.9 to 0.6 and the note is reworded to flag that the change is not covered by the gate, since an absence of tests is not the assurance a passing run provides. The result cache is now keyed per file path rather than per stem to avoid cross-package collisions, and a new TestPackageImportMatching test class adds coverage for qualified imports, dotted imports, package-relative imports, non-matching imports, and the reduced-confidence path