Skip to content

Add generic ITestHostLauncher extension point + PackagedApp reference extension#9454

Open
Evangelink wants to merge 17 commits into
mainfrom
evangelink-generic-testhost-launcher-rfc
Open

Add generic ITestHostLauncher extension point + PackagedApp reference extension#9454
Evangelink wants to merge 17 commits into
mainfrom
evangelink-generic-testhost-launcher-rfc

Conversation

@Evangelink

@Evangelink Evangelink commented Jun 26, 2026

Copy link
Copy Markdown
Member

Summary

Implements RFC 017 (reworked to be generic — see #9349). Adds a public, experimental MTP extension point — ITestHostLauncher — that lets an extension control how the out-of-process test host is launched, instead of the platform always calling Process.Start. The platform keeps owning everything around the launch (argument/environment preparation, the controller↔host IPC pipe, PID tracking, ITestHostProcessLifetimeHandler callbacks, and exit-code reconciliation) and delegates only the single "create and start the test host" step.

The hook is deliberately agnostic of the launch mechanism: the launcher does not have to start a local OS process. It can deploy and activate a packaged app, launch a container, or start the host on a remote machine. The returned ITestHostHandle exposes only lifecycle (WaitForExitAsync(CancellationToken), ExitCode, HasExited, Terminate, and IDisposable); there is no Exited event — WaitForExitAsync is the lifecycle mechanism. An opaque diagnostics-only string? Identifier is optional (it is intentionally not a numeric process id, so it can carry whatever a non-process launcher has — a container id, an AUMID-activated token, etc., or null).

Motivating scenario: testing packaged Windows apps (UWP and packaged WinUI), which must be deployed and AUMID-activated rather than Process.Start-ed — the blocker behind #2784.

What's included

Platform extension point (Microsoft.Testing.Platform)

  • New experimental public types (all [Experimental("TPEXP")], no init):
    • ITestHostLauncherTask<ITestHostHandle> LaunchTestHostAsync(TestHostLaunchContext, CancellationToken)
    • ITestHostHandle — generic, mechanism-agnostic (IDisposable); lifecycle via WaitForExitAsync(CancellationToken) / ExitCode / HasExited / Terminate; optional opaque string? Identifier for diagnostics (no Exited event, no numeric process id)
    • TestHostLaunchContextFileName, Arguments, EnvironmentVariables, WorkingDirectory
  • ITestHostControllersManager.AddTestHostLauncher(...) (factory + composite overloads).
  • Registering a launcher forces the controller (process-restart) host, so a run with only a launcher still launches out-of-process. At most one launcher is allowed (fails fast with a localized error).
  • TestHostControllersTestHost delegates the launch at the process.Start site and adapts the handle to the internal IProcess monitoring contract.
  • Identifier-less support: the premature-exit check is gated on HasExited only (not on the availability of any id), so a launcher that returns no identifier (container/remote/AUMID) is monitored purely through the handle lifecycle + the IPC PID handshake. No behavior change for the default Process.Start path.

Reference consumer (Microsoft.Testing.Extensions.PackagedApp)

  • A real, packable extension that consumes the hook for packaged Windows apps (UWP/WinUI share the same MSIX deploy + AUMID-activate mechanism — VSTest uses a single UwpTestHostRuntimeProvider for both). All public API is [Experimental("TPEXP")] and the package ships an experimental (alpha) version.
  • builder.AddPackagedAppDeployment() registers a launcher that deploys (stages the loose layout into an isolated directory) and launches the deployed copy, returning a handle whose Identifier is null — exercising the mechanism-agnostic path end-to-end. Packaged AUMID activation is scaffolded as clearly-marked follow-up.

Tests

  • Unit (TestApplicationBuilderTests): launcher forces process restart, singleton enforcement, duplicate-id validation. ✅
  • Acceptance (TestHostLauncherTests): a custom launcher drives the host end-to-end (with an identifier). ✅
  • Acceptance (PackagedAppDeploymentTests, Windows-gated): deploy-to-separate-directory + run succeeds with an identifier-less handle. ✅
  • Acceptance (PackagedApp.MSBuildRegistration): the package's build/buildTransitive props auto-register its TestingPlatformBuilderHook (build-time, asserted from the binlog; OS-agnostic). ✅

Notes

Related: #2784, #9349

Evangelink and others added 5 commits June 23, 2026 16:10
Introduce a public, experimental Microsoft.Testing.Platform extension point that
lets an extension control how the out-of-process test host is launched, replacing
the platform's default Process.Start. The abstraction is agnostic of the launch
mechanism (process, packaged/MSIX deploy+activate, container, remote): the launcher
returns an ITestHostHandle exposing only lifecycle (WaitForExitAsync, ExitCode,
HasExited, Exited, Terminate) with an optional ProcessId for diagnostics.

- New public types: ITestHostLauncher, ITestHostHandle, TestHostLaunchContext
  (all [Experimental(TPEXP)], no init accessors).
- ITestHostControllersManager.AddTestHostLauncher overloads + manager wiring;
  registering a launcher forces the controller host (RequireProcessRestart) and
  at most one launcher is allowed (localized OnlyOneTestHostLauncherSupported).
- TestHostControllersTestHost delegates the launch to the registered launcher and
  adapts the returned handle to the internal IProcess monitoring contract,
  tolerating a null PID for container/remote launches.
- Unit tests for restart-forcing, singleton, and duplicate-id validation.
- Acceptance test with a real consuming launcher proving end-to-end delegation.
- Rework RFC 017 to the generic ITestHostLauncher/ITestHostHandle shape and a
  package/deploy framing.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Prove the ITestHostLauncher hook works for a launch that is not a plain
Process.Start: add a real shipping extension Microsoft.Testing.Extensions.AppDeployment
that deploys (stages) the test host into an isolated directory, launches the
deployed copy, and returns an ITestHostHandle that exposes no local process id.

- Platform fix: the test host controller gated premature-exit on
  "testHostProcessId is null", which would reject a launcher that returns no PID
  (AUMID/container/remote). Gate on HasExited only; the real test host PID still
  arrives via the IPC handshake. No behavior change for the default Process.Start
  path (a null PID there always coincides with HasExited).
- New extension: AddAppDeployment, AppDeploymentLauncher, DeployedTestHostHandle
  (ProcessId => null), TestingPlatformBuilderHook, build props, PublicAPI, PACKAGE.md;
  added to TestFx.slnx; targets the SupportedNetFrameworks set.
- Acceptance test AppDeploymentTests references the packed package and asserts the
  host was deployed to a separate directory and the run succeeded with a PID-less
  handle.
- RFC updated to describe the HasExited-only gating.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The "AppDeployment" name was too generic: it over-promised a broad deployment
feature while the implementation only demonstrated the ITestHostLauncher hook.
Rename the consuming extension to Microsoft.Testing.Extensions.WinUI, anchoring it
to the concrete motivating scenario (#2784) instead.

- Microsoft.Testing.Extensions.WinUI: AddWinUIDeployment, WinUITestHostLauncher,
  WinUITestHostHandle (ProcessId => null), TestingPlatformBuilderHook (new GUID),
  build props, PublicAPI, PACKAGE.md; renamed in TestFx.slnx.
- The launcher is framed for WinUI: it implements the unpackaged deploy-and-launch
  path (stage the loose layout, launch the deployed app) and documents the packaged
  AUMID-activation branch as clearly-marked follow-up.
- Acceptance test renamed to WinUIDeploymentTests and gated to Windows; still proves
  end-to-end deploy + PID-less launch against the packed package.
- RFC non-goal updated: a reference WinUI consumer now exists (unpackaged path);
  packaged AUMID activation remains a separate follow-up.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
UWP and packaged WinUI are not distinct scenarios for launching the test host:
both produce MSIX packages and share the same deploy + AUMID-activate mechanism
(which is why VSTest exposes a single UwpTestHostRuntimeProvider for both). Naming
the extension after either app model is too narrow; name it after the shared
packaging format instead.

- Microsoft.Testing.Extensions.WinUI -> Microsoft.Testing.Extensions.Msix
  (AddMsixDeployment, MsixTestHostLauncher, MsixTestHostHandle, MsixExtensions).
- Casing is PascalCase "Msix" (not "MSIX"), matching .NET guidelines and the repo
  convention for 3+ letter acronyms (HtmlReport, TrxReport, CtrfReport).
- Docs/launcher text now describe UWP and packaged WinUI as the same MSIX mechanism;
  packaged AUMID activation remains a clearly-marked follow-up.
- Acceptance test renamed to MsixDeploymentTests (Windows-gated); still proves
  end-to-end deploy + PID-less launch against the packed package.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
"Msix" reads like a tool that creates MSIX packages; this extension is about
running tests *inside* a packaged Windows app. Name it after the scenario instead
of the package format.

- Microsoft.Testing.Extensions.Msix -> Microsoft.Testing.Extensions.PackagedApp
  (AddPackagedAppDeployment, PackagedAppTestHostLauncher, PackagedAppTestHostHandle,
  PackagedAppExtensions).
- Still covers both UWP and packaged WinUI (both ship as MSIX and share the same
  deploy + AUMID-activate mechanism); docs keep "MSIX" as the format acronym in prose
  while the package/API name describes the scenario.
- Acceptance test renamed to PackagedAppDeploymentTests (Windows-gated); still proves
  end-to-end deploy + PID-less launch against the packed package.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings June 26, 2026 11:17

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR implements RFC 017 by adding an experimental, public Microsoft.Testing.Platform (MTP) extension point (ITestHostLauncher + ITestHostHandle + TestHostLaunchContext) to allow extensions to control how the out-of-process test host is launched, and introduces a reference consumer extension (Microsoft.Testing.Extensions.PackagedApp) that exercises PID-less host monitoring.

Changes:

  • Added the experimental ITestHostLauncher launch hook (and related handle/context types) plus controller-manager registration and enforcement (single launcher, forces out-of-process controller host).
  • Updated TestHostControllersTestHost to delegate the launch step to a registered launcher and to support PID-less launchers by relying on lifecycle + IPC PID handshake.
  • Added acceptance/unit tests and a new packable extension (Microsoft.Testing.Extensions.PackagedApp) demonstrating deploy-to-isolated-directory + PID-less handle.
Show a summary per file
File Description
TestFx.slnx Adds the new PackagedApp extension project to the main solution.
test/UnitTests/Microsoft.Testing.Platform.UnitTests/TestApplicationBuilderTests.cs Adds unit tests validating launcher registration, singleton enforcement, and process-restart forcing.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/TestHostLauncherTests.cs Adds acceptance coverage for end-to-end custom launcher usage.
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/PackagedAppDeploymentTests.cs Adds Windows-gated acceptance coverage for PID-less launcher behavior via PackagedApp extension.
src/Platform/Microsoft.Testing.Platform/TestHostControllers/TestHostLaunchContext.cs Introduces the launch context passed to a custom launcher.
src/Platform/Microsoft.Testing.Platform/TestHostControllers/TestHostControllersManager.cs Adds registration/build pipeline for ITestHostLauncher and enforces single launcher.
src/Platform/Microsoft.Testing.Platform/TestHostControllers/TestHostControllerConfiguration.cs Carries the optional launcher through controller configuration.
src/Platform/Microsoft.Testing.Platform/TestHostControllers/ITestHostLauncher.cs Adds the experimental launcher interface contract.
src/Platform/Microsoft.Testing.Platform/TestHostControllers/ITestHostHandle.cs Adds the experimental handle contract used for monitoring launched hosts.
src/Platform/Microsoft.Testing.Platform/TestHostControllers/ITestHostControllerManager.cs Adds public controller-manager APIs to register a launcher (factory + composite).
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.zh-Hant.xlf Adds localized entry for single-launcher enforcement message.
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.zh-Hans.xlf Adds localized entry for single-launcher enforcement message.
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.tr.xlf Adds localized entry for single-launcher enforcement message.
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.ru.xlf Adds localized entry for single-launcher enforcement message.
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.pt-BR.xlf Adds localized entry for single-launcher enforcement message.
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.pl.xlf Adds localized entry for single-launcher enforcement message.
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.ko.xlf Adds localized entry for single-launcher enforcement message.
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.ja.xlf Adds localized entry for single-launcher enforcement message.
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.it.xlf Adds localized entry for single-launcher enforcement message.
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.fr.xlf Adds localized entry for single-launcher enforcement message.
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.es.xlf Adds localized entry for single-launcher enforcement message.
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.de.xlf Adds localized entry for single-launcher enforcement message.
src/Platform/Microsoft.Testing.Platform/Resources/xlf/PlatformResources.cs.xlf Adds localized entry for single-launcher enforcement message.
src/Platform/Microsoft.Testing.Platform/Resources/PlatformResources.resx Adds the OnlyOneTestHostLauncherSupported resource string.
src/Platform/Microsoft.Testing.Platform/PublicAPI/PublicAPI.Unshipped.txt Tracks the new experimental public APIs.
src/Platform/Microsoft.Testing.Platform/Hosts/TestHostControllersTestHost.cs Delegates host launch via ITestHostLauncher and supports PID-less launchers.
src/Platform/Microsoft.Testing.Platform/Helpers/System/TestHostHandleToProcessAdapter.cs Adapts public ITestHostHandle to internal IProcess.
src/Platform/Microsoft.Testing.Extensions.PackagedApp/TestingPlatformBuilderHook.cs Builder hook for MSBuild-driven extension registration.
src/Platform/Microsoft.Testing.Extensions.PackagedApp/PublicAPI/PublicAPI.Unshipped.txt Tracks new public APIs for the PackagedApp extension.
src/Platform/Microsoft.Testing.Extensions.PackagedApp/PublicAPI/PublicAPI.Shipped.txt Initializes shipped API tracking file for the new package.
src/Platform/Microsoft.Testing.Extensions.PackagedApp/PackagedAppTestHostLauncher.cs Implements deploy + launch using the new launcher extension point.
src/Platform/Microsoft.Testing.Extensions.PackagedApp/PackagedAppTestHostHandle.cs Implements a PID-less test host handle over a process.
src/Platform/Microsoft.Testing.Extensions.PackagedApp/PackagedAppExtensions.cs Adds AddPackagedAppDeployment() builder extension to register the launcher.
src/Platform/Microsoft.Testing.Extensions.PackagedApp/PACKAGE.md Documents the new experimental PackagedApp extension package.
src/Platform/Microsoft.Testing.Extensions.PackagedApp/Microsoft.Testing.Extensions.PackagedApp.csproj Adds the new PackagedApp extension project and packaging layout.
src/Platform/Microsoft.Testing.Extensions.PackagedApp/buildTransitive/Microsoft.Testing.Extensions.PackagedApp.props Wires transitive MSBuild import to the multi-targeting props.
src/Platform/Microsoft.Testing.Extensions.PackagedApp/buildMultiTargeting/Microsoft.Testing.Extensions.PackagedApp.props Declares the TestingPlatformBuilderHook MSBuild item for extension discovery.
src/Platform/Microsoft.Testing.Extensions.PackagedApp/build/Microsoft.Testing.Extensions.PackagedApp.props Wires non-transitive build import to the multi-targeting props.
src/Platform/Microsoft.Testing.Extensions.PackagedApp/BannedSymbols.txt Adds banned-symbols rules for the new PackagedApp extension project.
docs/RFCs/017-TestHost-Launcher.md Adds/updates the RFC describing the launcher design and scenarios.

Review details

  • Files reviewed: 40/40 changed files
  • Comments generated: 2
  • Review effort level: Low

Evangelink and others added 2 commits June 26, 2026 13:44
- TestHostHandleToProcessAdapter: give the "no process id" InvalidOperationException
  a descriptive message instead of an empty one, so the PID-less path is diagnosable.
- PackagedApp launcher: clean up the staged deployment directory once the host has
  exited (the handle now owns the directory and best-effort deletes it on Dispose),
  preventing temp-dir accumulation in CI. Update the acceptance test to no longer
  require the deployment directory to remain on disk after the run.
- Mirror the RFC review fixes (env-var wording, packaged-app example) in the doc
  shipped with the implementation.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…entifier

Per design review on the RFC:
- Remove the redundant Exited event from ITestHostHandle; consumers use
  WaitForExitAsync. The internal IProcess adapter synthesizes its informational
  Exited event from the exit task instead.
- Replace int? ProcessId with an optional free-form string Identifier (diagnostics
  only: PID, container id, remote host:pid, ...). The controller host logs it where
  the handle is visible; the adapter no longer pretends to expose a numeric PID.
- Update the PackagedApp handle (Identifier => null) and the acceptance asset handle
  (Identifier => process id string) accordingly, plus PublicAPI and the RFC doc.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@Evangelink Evangelink marked this pull request as ready for review June 26, 2026 12:46
Copilot AI review requested due to automatic review settings June 26, 2026 12:46
- PackagedAppTestHostHandle.Dispose: the two best-effort cleanup catches
  (IOException, UnauthorizedAccessException) now log via Debug.WriteLine instead of
  being empty, keeping cleanup non-fatal while satisfying the empty-catch rule.
- TestHostHandleToProcessAdapter.RaiseExitedWhenDoneAsync: replace the bare generic
  catch with catch (Exception ex) and a Debug.WriteLine, addressing the generic
  catch-clause finding while preserving the swallow-and-continue behavior for the
  informational Exited event.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 40/40 changed files
  • Comments generated: 1
  • Review effort level: Low

Evangelink and others added 2 commits June 26, 2026 14:54
…sthost-launcher-rfc

# Conflicts:
#	src/Platform/Microsoft.Testing.Platform/PublicAPI/PublicAPI.Unshipped.txt
ITestHostHandle has no Exited event (consumers use WaitForExitAsync); drop the
stale "Exited" from the lifecycle-contract comment so it matches the interface and
the adapter comment. Also spell "queryable".

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings June 26, 2026 13:18

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 40/40 changed files
  • Comments generated: 3
  • Review effort level: Low

@Evangelink Evangelink enabled auto-merge (squash) June 26, 2026 13:38
@Evangelink

This comment has been minimized.

…isposable

Per RFC design review:
- ITestHostHandle now extends IDisposable so the platform deterministically releases
  handle-held OS resources (it already wraps the handle in `using` via the adapter).
- WaitForExitAsync takes a CancellationToken (the runtime API and the repo's
  netstandard2.0 polyfill both support one). Threaded through the internal IProcess
  contract, SystemProcess, and the handle adapter.
- TestHostControllersTestHost passes its cancellation token when waiting for the host
  to exit; on cancellation it terminates the host and waits (uncancelable) for full
  exit so the existing exit-code reconciliation still observes a real OS exit code.
  The normal (non-canceled) path is unchanged.
- Pre-existing internal callers (Retry, HangDump) pass CancellationToken.None to keep
  their exact prior behavior; the adapter synthesizes its informational Exited event
  with a dispose-linked token.
- Document ExitCode-before-HasExited as undefined, clarify Quote/PasteArguments are
  placeholders, and fix the docker example so Terminate() tears down the container
  rather than only killing the local client.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Evangelink added a commit that referenced this pull request Jun 26, 2026
- ITestHostHandle now extends IDisposable and WaitForExitAsync takes a
  CancellationToken (both supported by the runtime and the netstandard2.0 polyfill).
  The platform disposes the handle after exit and passes its cancellation token while
  waiting, reconciling the real exit code afterwards.
- Document ExitCode-before-HasExited as undefined; note Quote/PasteArguments are
  placeholders for proper argument quoting; fix the container example so Terminate()
  tears down the container (docker stop) rather than only killing the local client.
- Record the design evolution in Alternatives.

Implemented in PR #9454.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@Evangelink

This comment has been minimized.

… WorkingDirectory

- Remove the PackagedApp launcher's write of a deployment-marker file into the
  original output directory (a shipping-side effect that could fail on read-only dirs
  and leave files behind). The acceptance test now proves deployment by having the
  deployed test host self-report its AppContext.BaseDirectory via a platform-forwarded
  env var, keeping the affordance in the test asset.
- Honor cancellation: ThrowIfCancellationRequested before the deploy and during the
  recursive copy.
- Honor TestHostLaunchContext.WorkingDirectory when set, defaulting to the deployment
  directory only when it is null.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings June 26, 2026 14:36

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 44/44 changed files
  • Comments generated: 3
  • Review effort level: Low

@Evangelink

This comment has been minimized.

Evangelink and others added 2 commits June 26, 2026 17:31
…egistration acceptance test

- Mark PackagedAppExtensions / AddPackagedAppDeployment [Experimental("TPEXP")] and
  prefix the PublicAPI entries with [TPEXP] (the MSBuild codegen hook stays non-experimental).
- Give the package an experimental (alpha) version via
  MicrosoftTestingExtensionsPackagedApp{VersionPrefix,PreReleaseVersionLabel} +
  SuppressFinalPackageVersion, matching the OpenTelemetry/CtrfReport convention.
- Update the buildTransitive props to the build/<tfm> forwarding pattern (#9431).
- Plumb MicrosoftTestingExtensionsPackagedAppVersion through AcceptanceTestBase and the
  PackagedApp deployment asset (NoWarn TPEXP/NETSDK1201).
- Add PackagedApp.MSBuildRegistration acceptance test asserting the build/buildTransitive
  props auto-register the TestingPlatformBuilderHook via the MSBuild self-registration codegen.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…nistic marker path; drop redundant IDisposable

- TestHostHandleToProcessAdapter: return from the catch in RaiseExitedWhenDoneAsync so the
  informational Exited event only fires on a genuine host exit, not when the wait faults or is
  cancelled (e.g. the adapter is disposed before the host exits).
- TestHostLauncherTests asset: write the LaunchTestHostAsync marker next to context.FileName so its
  location is deterministic regardless of the controller process working directory.
- PackagedAppTestHostHandle: ITestHostHandle already extends IDisposable, so drop the redundant
  IDisposable on the class declaration.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings June 26, 2026 15:43

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 46/46 changed files
  • Comments generated: 3
  • Review effort level: Low

@Evangelink

This comment has been minimized.

Evangelink and others added 2 commits June 26, 2026 18:09
…ackage packs on Linux/macOS

The Linux/macOS CI legs build and pack via NonWindowsTests.slnf (NonWindowsBuild=true);
Microsoft.Testing.Platform.slnf is the matching dev filter. Because the PackagedApp project was
not a member of either filter, its NuGet package was never produced on those legs, so
artifacts/packages/Debug/Shipping had no Microsoft.Testing.Extensions.PackagedApp.*.nupkg.

AcceptanceTestBase's static constructor calls ExtractVersionFromPackage("Microsoft.Testing.Extensions.PackagedApp.")
which throws when the package is absent, failing the type initializer and therefore EVERY acceptance
test in the assembly (523 failures on Linux Debug). Windows legs build the full TestFx.slnx, so the
package was already present there.

Add the project alongside OpenTelemetry (its experimental sibling) in both filters.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…nt asset

- TestHostControllersTestHost: when the wait is cancelled, the host may exit between cancellation
  and the Kill() call (Kill throws InvalidOperationException when there is no process left to
  terminate). Make termination best-effort by swallowing that exception; the subsequent
  WaitForExitAsync still reconciles the real exit code.
- PackagedAppDeploymentTests asset: collapse the two consecutive <NoWarn> entries into a single
  <NoWarn>$(NoWarn);TPEXP;NETSDK1201</NoWarn> to avoid any ambiguity about the effective value.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings June 26, 2026 16:13

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 48/48 changed files
  • Comments generated: 2
  • Review effort level: Low

Comment on lines +334 to +343
try
{
testHostProcess.Kill();
}
catch (InvalidOperationException)
{
// The host may have exited between the cancellation and this Kill call, in which
// case Kill throws because there is no longer a process to terminate. That is the
// outcome we wanted, so treat termination as best-effort and ignore it.
}
Comment on lines +8 to +12
/// <summary>
/// This class is used by Microsoft.Testing.Platform.MSBuild to hook into the Testing Platform Builder to add packaged Windows (UWP/WinUI) test host deployment support.
/// </summary>
public static class TestingPlatformBuilderHook
{
@Evangelink

Copy link
Copy Markdown
Member Author

🧪 Test quality grade — PR #9454

ΔTestGradeBandNotes
new PackagedAppDeploymentTests.
PackagedAppDeployment_
DeploysAndLaunchesTestHost_
WithoutLocalPid
A 90–100 Three complementary assertions — exit code, marker-file existence, and deployment-directory inequality — each verifying a distinct aspect of the deployment contract.
new PackagedAppMSBuildRegistrationTests.
PackagedApp_
TestingPlatformBuilderHook_
IsRegistered_
ViaBuildProps
A 90–100 Implicit .Single() guards serve as existence assertions; final Assert.Contains verifies the exact generated extension registration string.
new TestApplicationBuilderTests.
TestHostLauncher_
DuplicatedId_
ShouldFail
A 90–100 Exact exception type plus message content checks (duplicated ID and type name) — thorough failure-mode verification consistent with sibling tests.
new TestApplicationBuilderTests.
TestHostLauncher_
MultipleRegistered_
ShouldFail
A 90–100 No issues found. Exact InvalidOperationException check is complete for a multiple-launcher-registration rejection scenario.
new TestApplicationBuilderTests.
TestHostLauncher_
WhenRegistered_
ForcesProcessRestartAndIsStored
A 90–100 Two assertions verify both RequireProcessRestart is set and the launcher instance is the exact object stored in the configuration.
new TestHostLauncherTests.
CustomLauncher_
IsUsedToStartTestHost_
AndRunSucceeds
A 90–100 Exit code check plus marker-file content comparison prove the platform delegated launch to the custom ITestHostLauncher rather than using its default process start.

This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Re-run with
/grade-tests.

🤖 Automated content by GitHub Copilot. Posted via a maintainer's GitHub token, so it appears under their account — the account owner did not write or approve this content personally. Generated by the Grade Tests on PR (on open / sync) workflow. · 431.6 AIC · ⌖ 13.2 AIC · ⊞ 43.8K · [◷]( · )

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants