Multiple selections in test UI run individually/isolated

[Terrance]

Originally raised at redhat-developer/vscode-java#4338.

Observed behaviour: multiple methods under a Java test class (or multiple test classes under a package) selected in the Tests sidebar tree seem to run significantly slower than when just selecting the single parent class (or package).

This seems to be because each selection does a full setup/teardown (i.e. selecting a class does one setup/teardown cycle, before the first and after the last method; selecting four methods from that class does four cycles, one around each method). For projects like Spring web applications and their unit tests, the application is recreated unnecessarily for each test, and for large applications this causes a significant slowdown. Repeating failed unit tests from a previous run doesn't seem to be affected in this way.

My understanding is that JUnit at least doesn't provide much of a test selection interface, so I'm assuming there's a custom runner in play in order to target the selected or previously-failed tests in the first place.

[github-actions]

Hi Terrance (@Terrance), I'm an AI Support assistant here to help with your issue. While the team reviews your request, I wanted to provide some possible tips and documentation that might help you in the meantime.

It looks like the Java Test extension currently treats each selected test node as an individual run — spawning a new JVM (and thus full setup/teardown) per selection. There isn’t a built-in way yet to batch multiple method selections into a single class-level run, but you can:

• As a workaround, select the parent test class (or package) and use Run Tests, which will incur only one setup/teardown cycle for the whole class/package.
• Alternatively, if you only need to re-run failures, use Run Failed Tests — this uses a filtered run and avoids full teardown per test.

This feature is being tracked here:

• Be able to select multiple nodes from the explorer and run/debug
#874

Other references with low confidence

• (none at the moment)

The team will respond to your issue shortly. I hope these suggestions are helpful in the meantime. If this comment helped you, please give it a 👍. If the suggestion was not helpful or incorrect, please give it a 👎. Your feedback helps us improve!

[Terrance]

if you only need to re-run failures, use Run Failed Tests

Indeed this does also avoid multiple cycles, though more often that not I'm just wanting to hand-pick a set of individual tests relevant to a change, not replaying failures.

[wenytang-ms]

Hi Terrance (@Terrance) — thanks for the detailed report. I dug into the code path and wanted to share the root cause, because the fix is unfortunately less local than it first appears.
What an actual fix would require

To get a single JVM for an arbitrary multi-method selection we'd have to bypass the stock RemoteTestRunner protocol. The realistic option is to extend the in-tree custom launcher (com.microsoft.java.test.runner-jar-with-dependencies.jar, currently used for TestNG only) with a
JUnit Platform mode that:

• depends on junit-platform-launcher,
• builds a single LauncherDiscoveryRequest with multiple DiscoverySelectors.selectMethod(class, method) selectors, and
• streams events back over the existing socket protocol (mirroring the TestNG launcher).

Routing same-class (and ideally cross-class) multi-method batches through that new launcher in mergeTestMethods() + resolveLaunchConfigurationForRunner() would then collapse them into one JVM. JUnit 4 would still be limited unless we also wire up junit-vintage-engine in that
launcher.

[wenytang-ms]

The protocol PR is already merged, need jdtls to ship 3.8.100, then vscode-java-test may follow the new bits to ship the pre-release vscode-java-test