Update benchmarks to remove custom env for prefetcher - #996
Conversation
ankitaluthra1
commented
Aug 6, 2026
- Prefetcher is by default ON when cache_type is not SET, custom settings to use prefetcher with none is not needed anymore.
There was a problem hiding this comment.
Code Review
This pull request removes experimental read configurations from the benchmark Cloud Build pipeline and removes the explicit cache_type="none" parameter from several read operations in the performance microbenchmarks. The reviewer feedback highlights that removing cache_type="none" from random, mixed, and reopen read operations will cause them to default to readahead caching. This default behavior will trigger inefficient background prefetching and unnecessary GCS requests, which will introduce significant overhead and skew the benchmark results. It is recommended to retain cache_type="none" for these non-sequential read patterns.
| start_time = time.perf_counter() | ||
| files_it = itertools.cycle(file_paths) | ||
| open_kwargs = {"block_size": block_size, "cache_type": "none"} | ||
| open_kwargs = {"block_size": block_size} |
There was a problem hiding this comment.
Removing cache_type="none" from random reads (_read_op_rand) will cause them to use the default cache_type="readahead". For random read patterns, readahead caching is highly inefficient and will trigger unnecessary GCS requests, severely degrading benchmark performance and skewing the results. We should keep cache_type="none" for random reads.
| open_kwargs = {"block_size": block_size} | |
| open_kwargs = {"block_size": block_size, "cache_type": "none"} |
There was a problem hiding this comment.
Incorrect, the new default is none
| start_time = time.perf_counter() | ||
| files_it = itertools.cycle(file_paths) | ||
| open_kwargs = {"block_size": block_size, "cache_type": "none"} | ||
| open_kwargs = {"block_size": block_size} |
There was a problem hiding this comment.
Removing cache_type="none" from reopen reads (_read_op_reopen) will cause them to use the default cache_type="readahead". For reopen benchmarks (which repeatedly open a file, read a single chunk, and close it), readahead caching will trigger background prefetching that is immediately wasted when the file is closed, adding significant overhead and skewing the connection overhead measurements. We should keep cache_type="none" for reopen reads.
| open_kwargs = {"block_size": block_size} | |
| open_kwargs = {"block_size": block_size, "cache_type": "none"} |
There was a problem hiding this comment.
Incorrect, new default is none
| while time.perf_counter() - start_time < runtime: | ||
| path = next(files_it) | ||
| with gcs.open(path, "rb", cache_type="none") as f: | ||
| with gcs.open(path, "rb") as f: |
There was a problem hiding this comment.
Removing cache_type="none" from mixed reads (_read_op_mixed) will cause them to use the default cache_type="readahead". Since mixed reads include random seeks, readahead caching can introduce significant overhead and skew the benchmark results. It is better to keep cache_type="none" for mixed reads.
| with gcs.open(path, "rb") as f: | |
| with gcs.open(path, "rb", cache_type="none") as f: |
There was a problem hiding this comment.
Incorrect, new default is none
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #996 +/- ##
=======================================
Coverage 90.03% 90.03%
=======================================
Files 16 16
Lines 3593 3593
=======================================
Hits 3235 3235
Misses 358 358 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|