Makes prefetcher none cache default - #992
Conversation
There was a problem hiding this comment.
Code Review
This pull request enables the adaptive concurrent prefetcher by default, updating the default concurrency from 1 to 4 and dynamically setting the default cache type to "none" when prefetching is active (or "readahead" when disabled). The documentation and test suites have been updated to reflect these new defaults. Feedback on the changes suggests handling potential string values for the prefetching kwarg to avoid incorrect boolean evaluation, updating ZonalFile to align with the new dynamic cache defaults, and wrapping the environment variable integer parsing in a try-except block to prevent import-time crashes.
… include string value
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request enables adaptive concurrent prefetching by default, changing the default concurrency from 1 to 4 and setting the default cache type to "none" when prefetching is active. It also updates the documentation and test suite to reflect these new defaults. Regarding the feedback, a discrepancy was identified in how use_experimental_adaptive_prefetching is resolved in GCSFile.__init__ compared to ZonalFile.__init__. Specifically, passing None explicitly in kwargs would evaluate to False and disable prefetching while keeping cache_type="none", resulting in no caching or prefetching at all. A code suggestion has been provided to resolve this consistently.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #992 +/- ##
==========================================
+ Coverage 89.68% 90.03% +0.34%
==========================================
Files 16 16
Lines 3579 3593 +14
==========================================
+ Hits 3210 3235 +25
+ Misses 369 358 -11 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request enables the adaptive concurrent prefetcher by default with a default concurrency of 4 and a default cache type of "none". It refactors GCSFile and ZonalFile to dynamically resolve the cache type and prefetcher settings when not explicitly configured, and updates the documentation and test suite accordingly. Feedback on the changes highlights a potential RuntimeError or deadlock in gcsfs/prefetcher.py when scheduling the asynchronous close task on a running loop that is not the prefetcher's own background loop; a fix is suggested to ensure tasks are scheduled on the correct event loop.
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request enables adaptive concurrent prefetching by default for GCSFile and ZonalFile by resolving the default cache_type to None (which dynamically defaults to "none" with prefetching active). It also increases the default concurrency from 1 to 4. The review feedback correctly identifies two important issues: first, the accidental removal of the critical GoogleCredentials.load_tokens() call in gcsfs/core.py, and second, the need to validate that DEFAULT_GCSFS_CONCURRENCY is a positive integer to avoid invalid concurrency limits.
No description provided.