Forward Azure DevOps logging commands over the dotnet test pipe (multi-assembly)#9463
Draft
Evangelink wants to merge 1 commit into
Draft
Forward Azure DevOps logging commands over the dotnet test pipe (multi-assembly)#9463Evangelink wants to merge 1 commit into
Evangelink wants to merge 1 commit into
Conversation
…i-assembly) Under the dotnet test pipe protocol the host installs a no-op output device (the SDK's TerminalTestReporter owns user-facing output), so the AzureDevOpsReport extension's logging commands (##[group], ##vso[...]) were swallowed in multi-assembly runs. This adds protocol version 1.2.0 with a new AzureDevOpsLogMessage (serializer id 11): on an Azure DevOps agent with a negotiated version >= 1.2.0, those marked lines are forwarded to the SDK instead of dropped, while all other host output stays suppressed. Single-assembly behavior is unchanged. The SDK-side consumption (deserialize -> TerminalTestReporter.WriteMessage) is a separate follow-up. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates Microsoft.Testing.Platform’s dotnet test pipe protocol to selectively forward Azure DevOps logging commands (e.g., ##[group], ##[endgroup], ##vso[...]) during multi-assembly dotnet test runs, where the host currently installs a no-op output device and AzDO extension output is otherwise swallowed.
Changes:
- Introduces protocol 1.2.0 and a new wire message
AzureDevOpsLogMessage(serializer id 11) with version-gated sending (IsLogForwardingSupported). - Adds
DotnetTestPassthroughOutputDevice+AzureDevOpsCommandOutputDeviceDatamarker so only explicitly-marked AzDO command lines are forwarded under the pipe protocol (and only on AzDO agents). - Updates unit + acceptance tests and contract “tripwire” tests to lock down the new serializer id and protocol-version stability.
Show a summary per file
| File | Description |
|---|---|
| test/UnitTests/Microsoft.Testing.Platform.UnitTests/OutputDevice/DotnetTestPassthroughOutputDeviceTests.cs | Adds unit coverage for swallow/forward gating behavior in the passthrough device. |
| test/UnitTests/Microsoft.Testing.Platform.UnitTests/OutputDevice/AzureDevOpsLogIssueFormatterTests.cs | Adds tests to ensure TF_BUILD gating is independent from the platform opt-out. |
| test/UnitTests/Microsoft.Testing.Platform.UnitTests/IPC/ProtocolTests.cs | Adds AzureDevOpsLogMessage serializer round-trip tests; updates protocol/version stability assertions. |
| test/UnitTests/Microsoft.Testing.Platform.DotnetTestProtocolContract.UnitTests/DotnetTestProtocolContractTests.cs | Mirrors serializer-id and protocol-version stability checks in the shared contract test project. |
| test/UnitTests/Microsoft.Testing.Extensions.UnitTests/AzureDevOpsSummaryReporterTests.cs | Updates expectations to treat AzDO command output as TextOutputDeviceData (marker-compatible). |
| test/UnitTests/Microsoft.Testing.Extensions.UnitTests/AzureDevOpsLogGroupReporterTests.cs | Same as above for group open/close output. |
| test/UnitTests/Microsoft.Testing.Extensions.UnitTests/AzureDevOpsArtifactUploaderTests.cs | Same as above for artifact/tag command emission. |
| test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/DotnetTestPipe/DotnetTestPipeProtocol.cs | Extends the fake-SDK protocol decoder with serializer id 11 and message-body decoding. |
| test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/DotnetTestPipe/DotnetTestPipeBaselineTests.cs | Updates baseline expectations for host-advertised protocol versions (now includes 1.2.0). |
| test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/DotnetTestPipe/DotnetTestPipeAzureDevOpsForwardingTests.cs | New acceptance tests validating on-wire forwarding behavior under version + AzDO gating. |
| src/Platform/Microsoft.Testing.Platform/ServerMode/DotnetTest/IPC/Serializers/AzureDevOpsLogMessageSerializer.cs | Adds serializer implementation for the new AzureDevOpsLogMessage contract. |
| src/Platform/Microsoft.Testing.Platform/ServerMode/DotnetTest/IPC/ObjectFieldIds.cs | Adds AzureDevOpsLogMessageFieldsId with serializer id 11 and field ids. |
| src/Platform/Microsoft.Testing.Platform/ServerMode/DotnetTest/IPC/Models/AzureDevOpsLogMessage.cs | Adds the new IPC model representing forwarded AzDO log command lines. |
| src/Platform/Microsoft.Testing.Platform/ServerMode/DotnetTest/IPC/Constants.cs | Bumps supported protocol versions to 1.0.0;1.1.0;1.2.0 and documents the forwarding behavior. |
| src/Platform/Microsoft.Testing.Platform/ServerMode/DotnetTest/DotnetTestConnection.cs | Adds IsLogForwardingSupported and sends AzureDevOpsLogMessage frames when appropriate. |
| src/Platform/Microsoft.Testing.Platform/OutputDevice/OutputDeviceManager.cs | Installs DotnetTestPassthroughOutputDevice under pipe protocol on AzDO agents; keeps no-op otherwise. |
| src/Platform/Microsoft.Testing.Platform/OutputDevice/DotnetTestPassthroughOutputDevice.cs | Implements the selective forwarding output device (marker-based forwarding + version gate). |
| src/Platform/Microsoft.Testing.Platform/OutputDevice/AzureDevOpsLogIssueFormatter.cs | Introduces IsAzureDevOpsAgent (TF_BUILD-only) and refactors IsAzureDevOpsEnvironment to use it. |
| src/Platform/Microsoft.Testing.Platform/OutputDevice/AzureDevOpsCommandOutputDeviceData.cs | Adds a marker output type used to identify AzDO command lines for forwarding. |
| src/Platform/Microsoft.Testing.Platform/IPC/Serializers/RegisterSerializers.cs | Registers the new serializer so it participates in the named-pipe protocol. |
| src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsSummaryReporter.cs | Switches command emission to the marker data type so it can be forwarded in multi-assembly pipe runs. |
| src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsSlowTestReporter.cs | Same marker switch for slow-test logging lines. |
| src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsReporter.cs | Same marker switch for log-issue/build-tag commands emitted by the reporter. |
| src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsLogGroupReporter.cs | Same marker switch for ##[group]/##[endgroup] emission. |
| src/Platform/Microsoft.Testing.Extensions.AzureDevOpsReport/AzureDevOpsArtifactUploader.cs | Same marker switch for artifact-upload/build-tag commands. |
Review details
Comments suppressed due to low confidence (1)
test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/DotnetTestPipe/DotnetTestPipeBaselineTests.cs:36
- The test name still references advertising protocol versions 1.0.0 and 1.1.0, but this file now asserts the host advertises up to 1.2.0. Renaming the test to match the current contract would avoid confusion when reading failures or test output.
[TestMethod]
public async Task DotnetTestPipe_TestAppAdvertises100And110_NegotiatesDownToV100WithOldSdk()
{
- Files reviewed: 25/25 changed files
- Comments generated: 0
- Review effort level: Low
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
| break; | ||
|
|
||
| case AzureDevOpsLogMessage azureDevOpsLogMessage: | ||
| await _dotnetTestPipeClient.RequestReplyAsync<AzureDevOpsLogMessage, VoidResponse>(azureDevOpsLogMessage, _cancellationTokenSource.CancellationToken).ConfigureAwait(false); |
|
|
||
| return response.Properties?.TryGetValue(HandshakeMessagePropertyNames.SupportedProtocolVersions, out string? protocolVersion) == true && | ||
| IsVersionCompatible(protocolVersion, supportedProtocolVersions); | ||
| if (response.Properties?.TryGetValue(HandshakeMessagePropertyNames.SupportedProtocolVersions, out string? protocolVersion) == true) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Makes the
Microsoft.Testing.Extensions.AzureDevOpsReportextension's logging commands (##[group],##[endgroup],##vso[...]) work in multi-assemblydotnet testruns, where they were previously swallowed.Why
Under the
dotnet testpipe protocol (--server dotnettestcli), the host installs a no-op output device (OutputDeviceManager.BuildAsync) because the SDK'sTerminalTestReporterowns all user-facing output (see #7161 / dotnet/sdk#51615). Every AzDO reporter emits throughIOutputDevice, so in multi-assembly runs all of its output — log groups,##vso[task.logissue]failure annotations, slow-test lines, artifact-upload/build-tag commands, summary commands — was dropped on the floor. Single-assembly runs (real terminal) were unaffected.How
Selective passthrough over the pipe protocol, gated so old SDKs are never affected:
AzureDevOpsLogMessage(serializer id 11, fieldsExecutionId/InstanceId/LogText) added to the dotnet-test wire contract;ProtocolConstants.SupportedVersionsbumped to1.0.0;1.1.0;1.2.0.ObjectFieldIds.cs/Constants.csare the source-shared contract (kept aligned with dotnet/sdk).DotnetTestConnection.IsLogForwardingSupportedis true only when the negotiated version is ≥ 1.2.0, so the host never sends id 11 to a 1.0.0/1.1.0 SDK.AzureDevOpsCommandOutputDeviceData : TextOutputDeviceData. Because it derives fromTextOutputDeviceData(notFormattedTextOutputDeviceData), single-assembly rendering routes through the existingcase TextOutputDeviceDataand is byte-for-byte unchanged.DotnetTestPassthroughOutputDevice— under the pipe protocol on an Azure DevOps agent, this replaces the no-op device: it forwards only marked lines via the protocol and swallows everything else (preserving the deliberate suppression). It gates onTF_BUILDonly (newAzureDevOpsLogIssueFormatter.IsAzureDevOpsAgent), not theTESTINGPLATFORM_AZDO_OUTPUTopt-out — that opt-out is scoped to the platform's automatic##vso[task.logissue]emission, and single-assembly extension output is gated onTF_BUILDalone, so multi-assembly must match.Zero new public API (all new types are internal; the extension already has
InternalsVisibleTo).Tests
AzureDevOpsLogMessageserializer round-trip (incl. null fields);DotnetTestPassthroughOutputDeviceswallow behavior;IsAzureDevOpsAgentopt-out independence; theSerializerIds_AreStable/ProtocolVersion_IsStablewire tripwires (and their mirror in the standalone contract test project) updated for id 11 and 1.2.0.DotnetTestPipe/DotnetTestPipeAzureDevOpsForwardingTests): black-box over the wire via the fake-SDK harness — forwards##[group]/##[endgroup](with matchingInstanceId) at 1.2.0 + AzDO; does not forward at 1.1.0; does not forward off an AzDO agent. Existing pipe baseline tests updated for the version bump.All green locally: 9/9 DotnetTestPipe acceptance, platform unit + contract tripwires, 158 AzDO extension unit tests.
Out of scope / follow-ups
dotnet testSDK must add the aligned id-11 deserializer + replyVoidResponse, then writeLogTextto itsTerminalTestReporter— and that must land before any SDK advertises 1.2.0. This PR is safe in the meantime (forwarding is version-gated).InstanceIdlets the SDK scope##[group]/##[endgroup]per assembly.WarningMessageOutputDeviceDataemits remain swallowed in multi-asm (unchanged behavior; not targeted here).