Add test coverage for ResultsComparer and CLI helper paths#5199
Add test coverage for ResultsComparer and CLI helper paths#5199LoopedBard3 wants to merge 11 commits intodotnet:mainfrom
Conversation
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Adds new unit tests and small robustness tweaks across tooling to improve confidence in ResultsComparer output/CLI parsing and reduce platform-specific test failures.
Changes:
- Added a new
ResultsComparer.Testsproject with coverage forProgram,Data,Helper, andStats. - Updated ResultsComparer logic for filter handling, moniker parsing, and equivalence-test conclusions.
- Improved platform gating in
Startup.Testsand strengthened/nullability-corrected tests in the BenchmarkDotNet.Extensions harness.
Reviewed changes
Copilot reviewed 15 out of 15 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| src/tools/ScenarioMeasurement/Startup.Tests/StartupTests.cs | Updates Windows/Linux platform detection and adds an admin-privilege gate for ETW-related tests. |
| src/tools/ResultsComparer/TwoInputsComparer.cs | Treats EquivalenceTestConclusion.Base as non-difference (like Same) when filtering results. |
| src/tools/ResultsComparer/ResultsComparer.sln | Adds the new ResultsComparer.Tests project and additional solution configurations. |
| src/tools/ResultsComparer/Properties/AssemblyInfo.cs | Exposes internals to ResultsComparer.Tests via InternalsVisibleTo. |
| src/tools/ResultsComparer/Program.cs | Makes filter parsing resilient to null filter arrays. |
| src/tools/ResultsComparer/Data.cs | Adjusts moniker detection (including empty-key guard and Contains-based matching for newer TFMs). |
| src/tools/ResultsComparer.Tests/StatsTests.cs | Adds unit tests for Stats formatting/aggregation and validation behavior. |
| src/tools/ResultsComparer.Tests/ResultsComparerTestData.cs | Adds a JSON test-data generator for BDN-like inputs. |
| src/tools/ResultsComparer.Tests/ResultsComparer.Tests.csproj | Introduces the new ResultsComparer unit test project. |
| src/tools/ResultsComparer.Tests/ProgramTests.cs | Adds CLI-level tests validating error output and “no diffs” behavior. |
| src/tools/ResultsComparer.Tests/HelperTests.cs | Adds tests for file discovery and JSON deserialization helpers. |
| src/tools/ResultsComparer.Tests/DataTests.cs | Adds tests for nested zip + tar.gz extraction and dedup/version preference logic. |
| src/tests/harness/BenchmarkDotNet.Extensions.Tests/PartitionFilterTests.cs | Makes parsed config nullable-aware and asserts non-null on success. |
| src/tests/harness/BenchmarkDotNet.Extensions.Tests/CommandLineOptionsTests.cs | Adds more test cases for parsing/removal helpers (strings/bool/int). |
| src/harness/BenchmarkDotNet.Extensions/CommandLineOptions.cs | Fixes parsing when the target string parameter switch is missing. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 18 out of 18 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 19 out of 19 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 19 out of 19 changed files in this pull request and generated no new comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 19 out of 19 changed files in this pull request and generated no new comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 22 out of 22 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| var tarEntry = new UstarTarEntry(TarEntryType.RegularFile, entryName) | ||
| { | ||
| DataStream = new MemoryStream(Encoding.UTF8.GetBytes(content)) |
There was a problem hiding this comment.
In CreateTarGzArchive, each TarEntry's DataStream is set to a new MemoryStream but never disposed. Even though this is test-only code, it’s easy to avoid leaking disposables by creating the MemoryStream in a using scope and disposing it after TarWriter.WriteEntry returns.
| var tarEntry = new UstarTarEntry(TarEntryType.RegularFile, entryName) | |
| { | |
| DataStream = new MemoryStream(Encoding.UTF8.GetBytes(content)) | |
| using var dataStream = new MemoryStream(Encoding.UTF8.GetBytes(content)); | |
| var tarEntry = new UstarTarEntry(TarEntryType.RegularFile, entryName) | |
| { | |
| DataStream = dataStream |
Summary
This expands automated coverage in the repo’s tooling-focused test surface without large production refactors.
What changed
Added a new
ResultsComparer.Testsproject covering:Expanded
BenchmarkDotNet.Extensionscommand-line option tests to cover:Improved
Startup.Testsreliability by skipping ETW-dependent Windows tests when not running elevated, instead of aborting the runSmall correctness fixes found while adding coverage
The new tests exposed a few edge-case bugs, which are fixed in this PR:
CommandLineOptions.ParseAndRemoveStringsParameternow handles missing switches safelyResultsComparer.Programnow tolerates a missing optional--filterTwoInputsComparernow treatsEquivalenceTestConclusion.Baseas a no-diff caseResultsComparer.Datamoniker parsing is now null-safe for additional archive path shapesValidation
Validated the affected suites in the available environment:
ResultsComparer.TestspassingBenchmarkDotNet.Extensionscommand-line tests passingStartup.Testsnow skip environment-dependent cases cleanly