Skip to content

perf(files): reuse file stats for size filtering and adaptive concurrency - #182

Merged
luojiyin1987 merged 1 commit into
masterfrom
perf/reuse-file-stats
Sep 1, 2026
Merged

perf(files): reuse file stats for size filtering and adaptive concurrency#182
luojiyin1987 merged 1 commit into
masterfrom
perf/reuse-file-stats

Conversation

@luojiyin1987

Copy link
Copy Markdown
Contributor

Summary

  • Read file metadata once when size filtering or adaptive concurrency needs it.
  • Reuse retained file metadata for adaptive concurrency.
  • Keep the existing stat concurrency limit of 128.

The duplicate scan only occurs when both --max-file-size and --threads auto are enabled.

Preserved behavior

  • Size filtering runs before adaptive concurrency.
  • Filtered large files do not reduce the worker count.
  • Fixed threads without a size limit perform no metadata scan.
  • Stat failures still propagate to the caller.

Benchmark

Each value is the median of seven warm runs on local ext4.

Files Before After
1k 55.7 ms 32.4 ms
10k 560 ms 313 ms
50k 2.82 s 1.57 s

The measured cost scales approximately linearly with the file count.

Validation

  • npm run lint
  • npm test -- --runInBand
  • 26 test suites passed.
  • 222 tests passed.
  • git diff --check

Security

  • This change does not include logs or secrets.
  • No redaction was required.

@luojiyin1987
luojiyin1987 merged commit 8cd4847 into master Sep 1, 2026
6 checks passed
@luojiyin1987
luojiyin1987 deleted the perf/reuse-file-stats branch September 1, 2026 09:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant