Skip to content

perf: Avoid repeated test discovery for resolved test items - #1916

Merged
wenyt (wenytang-ms) merged 7 commits into
mainfrom
wenyt/avoid-repeated-test-discovery
Aug 26, 2026
Merged

perf: Avoid repeated test discovery for resolved test items#1916
wenyt (wenytang-ms) merged 7 commits into
mainfrom
wenyt/avoid-repeated-test-discovery

Conversation

@wenytang-ms

Copy link
Copy Markdown
Contributor

Summary

  • reuse synchronized Project and Class children after their first successful resolution
  • coalesce concurrent requests for the same test item and leave cancelled resolutions retryable
  • force subtree re-resolution for manual refreshes and classpath updates

This reduces repeated discovery work on subsequent runs. It does not change the initial JUnit 5 discovery cost in the language server, which is tracked separately in eclipse-jdtls/eclipse.jdt.ls#3870.

Related to #1915

Testing

  • npm run compile
  • npm run lint
  • added coverage for Project/Class reuse, concurrent resolution, forced invalidation, and cancellation
  • npm test: the new tests pass; the suite still reports the same 12 pre-existing failures from unavailable Java delegate handlers and the existing enqueueTestCases export mismatch

@wenytang-ms wenyt (wenytang-ms) changed the title Avoid repeated test discovery for resolved test items perfect:Avoid repeated test discovery for resolved test items Aug 24, 2026
@wenytang-ms wenyt (wenytang-ms) changed the title perfect:Avoid repeated test discovery for resolved test items perf:Avoid repeated test discovery for resolved test items Aug 24, 2026
@wenytang-ms wenyt (wenytang-ms) changed the title perf:Avoid repeated test discovery for resolved test items perf: Avoid repeated test discovery for resolved test items Aug 24, 2026
@wenytang-ms
wenyt (wenytang-ms) requested a balanced review from Copilot August 24, 2026 08:36

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Optimizes test discovery by caching resolved test-item children while supporting explicit invalidation.

Changes:

  • Coalesces concurrent resolutions and tracks invalidation versions.
  • Reuses resolved project/class children.
  • Forces refresh after manual and classpath updates, with new tests.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
src/controller/testController.ts Adds resolution caching, coalescing, and invalidation.
src/controller/testItemDataCache.ts Tracks resolution versions.
src/controller/utils.ts Invalidates and synchronizes document test items.
src/commands/testExplorerCommands.ts Forces discovery during refreshes.
test/suite/testController.loadChildren.test.ts Tests resolution reuse and cancellation.
test/suite/controllerUtils.updateItemForDocument.test.ts Tests document-update invalidation.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/controller/utils.ts
Comment thread src/controller/utils.ts Outdated
Comment thread src/controller/testController.ts Outdated

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.

Suppressed comments (2)

src/controller/testController.ts:109

  • If this project resolution is invalidated and there is no concurrent waiter, loadChildren completes successfully while the project remains unresolved. expandTests immediately traverses the current children after awaiting it, so a document update racing initial discovery can leave the run with only a partial project tree and omit tests. Keep the outer loadChildren call retrying the latest version unless its token was cancelled.
        if (token?.isCancellationRequested || resolutionVersion !== getResolutionVersion(item)) {
            return;

src/controller/testController.ts:121

  • A class resolution invalidated by a concurrent forced subtree refresh also returns as if resolution succeeded. Its caller can then continue with missing or stale methods even though canResolveChildren is still true. Retry the current class version in the outer loadChildren loop unless cancellation caused the return.
        if (token?.isCancellationRequested || resolutionVersion !== getResolutionVersion(item)) {
            return;

Comment thread src/controller/testController.ts Outdated
@wenytang-ms

wenyt (wenytang-ms) commented Aug 25, 2026

Copy link
Copy Markdown
Contributor Author

Local performance validation

Validated the latest PR commit 8030a0f against the full magefree/mage multi-module workspace.

Method

The benchmark used the same VS Code session for both measurements:

  1. Expand the unresolved mage-tests project and wait for org.mage.test.AI.basic to become visible.
  2. Collapse mage-tests and verify that the package is no longer visible.
  3. Expand the already-resolved project again and wait for the same package.

The collapse assertion ensures the cached result is not measuring a project that was already expanded. All 10 automation steps passed.

Results

Scenario Expand action Wait for package End-to-end
Cold project discovery 1.026 s 162.197 s 163.223 s
Cached project expansion 0.999 s 1.424 s 2.423 s
  • 67.4x faster on the second expansion
  • 98.5% reduction in end-to-end time
  • The automation includes screenshots and a fixed 1-second wait, so the cached resolution itself is expected to be below one second; 2.423 seconds is the UI automation upper bound.
  • This is consistent with the previous 6f60727 run (166.130 s cold / 2.955 s cached). The 1.7% cold difference and 0.532-second cached difference are within expected run/UI automation variation.

Relevant machine-readable step output:

PASS cold-expand                  1,026 ms
PASS cold-wait-for-package      162,197 ms
PASS collapse-resolved-project      695 ms
PASS verify-project-collapsed     1,342 ms
PASS cached-expand                  999 ms
PASS cached-wait-for-package      1,424 ms

Environment

  • VS Code 1.134.0
  • Language Support for Java 1.56.2026082208 pre-release
  • Debugger for Java 0.59.2026072407
  • Local Test Runner for Java 0.46.0 VSIX built from 8030a0f

This validates that the Project-resolution cache remains effective after the latest concurrency and metadata-consistency changes. As expected, the first discovery remains expensive; batch Class/method discovery is separate follow-up work.

Benchmark packaging note: the local VSIX manifest was aligned with the JAR filenames already present in the checked-in server directory so the local JDT bundle could load. No additional Test Runner runtime/source change was included in the measurement.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

src/controller/utils.ts:246

  • findTestTypesAndMethods can return a still-testable class with children: null after its last test method is removed: the Java model leaves JavaTestItem.children unset when there are no children. synchronizeItemsRecursively skips that falsy value, retaining the old method items, and this then marks the class resolved so those stale methods are reused indefinitely. Normalize missing children to an empty list before marking the class resolved.
            markTestClassesResolvedRecursively(testTypeItem);

Comment thread src/controller/testController.ts

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

src/controller/testController.ts:82

  • A caller with its own cancellation token can get stuck here behind a shared discovery started by another caller without that token. For example, cancelling a test run while it is coalesced with the Test Explorer's long-running project scan will not let the run return until that scan finishes. Race the shared promise with this caller's cancellation event, while leaving the shared promise in the map for its owner and other waiters.
            await pendingResolution;

src/controller/testController.ts:72

  • The file-deletion watcher removes items directly but does not invalidate the owning project's resolution version. If a project discovery computed its result before the deletion and completes afterward, it can reinsert the deleted class and mark the project resolved; this new early return then makes that stale class persist on subsequent runs. Invalidate the project (and relevant in-flight class resolutions) in onDidDelete before mutating the tree, as the document-update path now does.
    } else if (!item.canResolveChildren) {
        return;

@wenytang-ms
wenyt (wenytang-ms) marked this pull request as ready for review August 25, 2026 04:05

@chagong Changyong Gong (chagong) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One cache-invalidation race remains in the file-deletion path; details are inline.

Comment thread src/types.ts
Comment thread src/controller/testController.ts
@wenytang-ms
wenyt (wenytang-ms) merged commit 6758803 into main Aug 26, 2026
6 checks passed
@wenytang-ms
wenyt (wenytang-ms) deleted the wenyt/avoid-repeated-test-discovery branch August 26, 2026 05:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants