Code coverage

[spetersenms]

Code Coverage Feature — Reviewer Guide

Overview

This PR adds opt-in code coverage collection to AL-Go for GitHub. When a user sets enableCodeCoverage: true in their AL-Go settings, the pipeline collects line-level coverage data during test execution and outputs it in Cobertura XML format — the industry standard supported by tools like SonarQube, Codecov, and Azure DevOps.

The feature is off by default and preview. When disabled, zero code paths are affected. The only overhead is an unconditional MergeCoverage workflow job (~20s) that no-ops via hashFiles() gating when no coverage files exist.

---

Architecture: What's New vs. What's Ported

The TestRunner module is split into two distinct categories of code. Understanding this split is the key to an efficient review.

Ported from the BC Platform Test Runner (~60% of TestRunner code)

The BC platform ships an internal test runner used for running AL tests against Business Central containers via client services. The original lives at NAV\BuildArtifacts\w1Development\ALTestRunner\. This PR ports that code into AL-Go with architectural refactoring but minimal functional changes. The core test execution logic (connecting to containers, opening test forms, running tests, collecting results) is unchanged.

These files need less review scrutiny — they are a known-good module that ships with BC and is used in production daily:

AL-Go File	Original File	Similarity	What Changed
Internal/ALTestRunnerInternal.psm1	Internal/ALTestRunnerInternal.psm1	~45% line-for-line, but 100% functional	Decomposed into 5 smaller modules (see below). No logic changes — only extraction.
Internal/BCPTTestRunnerInternal.psm1	Internal/BCPTTestRunnerInternal.psm1	~95%	Typo fix (Setup-Enviroment → Setup-Environment), error headers added
Internal/ClientContext.ps1	Internal/ClientContext.ps1	~85%	Added explicit $disableSSL constructor parameter for PS7 support
Internal/AadTokenProvider.ps1	Internal/AadTokenProvider.ps1	~98%	Error headers added, otherwise identical
Internal/TestRunnerInternalForAIT.psm1	Internal/TestRunnerInternalForAIT.psm1	~90%	Documentation added, all 20 functions preserved
ALTestRunner.psm1	ALTestRunner.psm1	~80%	Added JUnit format support (was XUnit only), result formatting moved to new module
Internal/*.dll (4 files)	Internal/*.dll	100%	Identical binaries

Modules extracted from the original ALTestRunnerInternal.psm1 (these are the same functions, just in separate files now):

New Module	Functions Extracted	Purpose
Internal/Constants.ps1	N/A (was inline constants)	Centralized configuration constants
Internal/ModuleInit.ps1	DLL loading logic from module init block	DLL loading + WCF dependency management for PS7
Internal/ClientSessionManager.psm1	Open-ClientSessionWithWait, Open-ClientSession, SSL functions	Session lifecycle
Internal/TestFormHelpers.psm1	16 Set-*/Open-*/Clear-* functions	BC test form UI controls
Internal/CoverageCollector.psm1	CollectCoverageResults, SaveCodeCoverageMap	Coverage data collection from BC

Entirely New Code (~40% of TestRunner code)

These modules have no equivalent in the original BC test runner. These need full review attention:

Module	Lines	Purpose
TestResultFormatter.psm1	~246	Formats test results as JUnit or XUnit XML (original only had XUnit)
CoverageProcessor/CoverageProcessor.psm1	~490	Orchestrates BC coverage → Cobertura conversion
CoverageProcessor/ALSourceParser.psm1	~546	Parses AL source files to determine executable lines (coverage denominator)
CoverageProcessor/BCCoverageParser.psm1	~431	Parses BC coverage data files (CSV and XML formats)
CoverageProcessor/CoberturaFormatter.psm1	~427	Generates Cobertura XML from parsed data
CoverageProcessor/CoverageUtilities.ps1	~20	Shared utility (Test-PropertyExists)
RunPipeline.psm1	~200	New-ALTestRunnerOverride function (the override scriptblock)

New Actions

Action	Purpose
BuildCodeCoverageSummary/	Reads cobertura.xml from a build job and writes a markdown summary to the GitHub Step Summary
MergeCoverageSummaries/	Merges coverage from multiple build jobs into a single consolidated Cobertura XML and summary

---

How It Works: The Override Mechanism

Why an override is needed

AL-Go uses BcContainerHelper's Run-ALPipeline to build and test apps. Normally, Run-ALPipeline runs tests inside the BC container via its own internal test runner. However, code coverage collection requires a host-side orchestration approach — connecting to the container's client services endpoint, configuring coverage tracking on the test form, running tests, then pulling coverage data back.

BcContainerHelper doesn't support this natively. It does, however, support a RunTestsInBcContainer parameter that accepts a custom scriptblock to replace the default test execution.

The flow

User enables enableCodeCoverage: true
                    │
                    ▼
    ┌─────────────────────────────────┐
    │   RunPipeline.ps1 (line 465)   │
    │   Checks settings, reads       │
    │   codeCoverageSetup config     │
    └─────────────┬───────────────────┘
                  │
                  ▼
    ┌─────────────────────────────────┐
    │   RunPipeline.psm1             │
    │   New-ALTestRunnerOverride()   │
    │   Creates a .GetNewClosure()   │
    │   scriptblock capturing:       │
    │   - BuildArtifactFolder        │
    │   - TrackingType               │
    │   - ProduceMap                 │
    └─────────────┬───────────────────┘
                  │
                  ▼
    ┌─────────────────────────────────┐
    │   Run-ALPipeline               │
    │   (BcContainerHelper)          │
    │                                │
    │   For each test app:           │
    │   Calls the override           │
    │   scriptblock instead of       │
    │   its default test runner      │
    └─────────────┬───────────────────┘
                  │
                  ▼
    ┌─────────────────────────────────┐
    │   Override scriptblock         │
    │   (inside RunPipeline.psm1)    │
    │                                │
    │   1. Gets container URL via    │
    │      Get-BcContainerServer-    │
    │      Configuration             │
    │   2. Constructs service URL    │
    │      with tenant parameter     │
    │   3. Creates CodeCoverage      │
    │      output directory          │
    │   4. Calls Run-AlTests with    │
    │      coverage parameters       │
    │   5. Parses test results XML   │
    │      to determine pass/fail    │
    │   6. Handles append mode for   │
    │      multi-app result merging  │
    └─────────────┬───────────────────┘
                  │
                  ▼
    ┌─────────────────────────────────┐
    │   Run-AlTests                  │
    │   (ALTestRunner.psm1)          │
    │                                │
    │   Connects to BC container     │
    │   via client services (WCF),   │
    │   opens test form, configures  │
    │   coverage tracking, runs      │
    │   tests one by one, collects   │
    │   coverage .dat files          │
    └─────────────┬───────────────────┘
                  │
                  ▼
    ┌─────────────────────────────────┐
    │   Post-test processing         │
    │   (RunPipeline.ps1 line 762)   │
    │                                │
    │   1. Finds .dat files in       │
    │      CodeCoverage folder       │
    │   2. Imports CoverageProcessor │
    │   3. Converts .dat files to    │
    │      cobertura.xml             │
    │   4. Saves coverage stats JSON │
    └─────────────┬───────────────────┘
                  │
                  ▼
    ┌─────────────────────────────────┐
    │   Workflow steps               │
    │                                │
    │   _BuildALGoProject.yaml:      │
    │   • Upload CodeCoverage        │
    │     artifact per project       │
    │   • BuildCodeCoverageSummary   │
    │     → per-project markdown     │
    │                                │
    │   CICD.yaml / PRHandler.yaml:  │
    │   • MergeCoverage job          │
    │     downloads all coverage     │
    │     artifacts, merges XML,     │
    │     publishes consolidated     │
    │     MergedCodeCoverage         │
    └─────────────────────────────────┘

Custom override compatibility

If a user already has a custom RunTestsInBcContainer.ps1 override in their .AL-Go folder, AL-Go detects this and emits a warning instead of overriding. The user's custom script takes precedence. They can still use coverage by calling Run-AlTests directly in their custom override with the appropriate coverage parameters.

---

Coverage Processing Pipeline (New Code)

The coverage processing converts BC's proprietary format to industry-standard Cobertura XML. This is the most substantial new code in the PR.

Data flow

BC Coverage .dat files (CSV/XML from XMLport 130470/130007)
        │
        ▼
BCCoverageParser.psm1
  Read-BCCoverageFile() → parses CSV/XML entries
  Group-CoverageByObject() → groups by (ObjectType, ObjectId)
        │
        ▼
ALSourceParser.psm1
  Get-ALObjectMap() → scans .al source files for object definitions
  Get-ALExecutableLines() → determines which lines are executable code
  Get-ALProcedures() → extracts procedure boundaries for method-level mapping
        │
        ▼
CoverageProcessor.psm1
  Convert-BCCoverageToCobertura() → orchestrates the full conversion
  - Matches coverage data to source objects
  - Calculates line rates (covered/total)
  - Applies exclude patterns (e.g., *.PermissionSet.al)
        │
        ▼
CoberturaFormatter.psm1
  New-CoberturaDocument() → generates Cobertura XML document
  - Packages/classes/methods/lines hierarchy
  - Line-rate calculations at every level
  - Source file path mapping
        │
        ▼
cobertura.xml + cobertura.stats.json

Multi-job merge

When a repo has multiple AL-Go projects (each built in a separate job), the MergeCoverageSummaries action merges their individual cobertura.xml files using union semantics:

• Lines present in multiple files: takes the maximum hit count
• Lines present in only one file: included as-is
• Method-level detail is not preserved across merges (by design — line-level is sufficient)

---

Changes to Existing AL-Go Files

File	Change	Impact
ReadSettings.psm1	Added enableCodeCoverage (default: false) and codeCoverageSetup defaults	Settings framework
settings.schema.json	Added schema for both new settings	Schema validation
RunPipeline.ps1	~35 lines: reads coverage settings, imports override module, injects override or warns	Core pipeline
RunPipeline.ps1	~70 lines: post-test coverage processing (.dat → Cobertura conversion)	Core pipeline
CalculateArtifactNames/CalculateArtifactNames.ps1	Added CodeCoverage to artifact name list	Artifact naming
CalculateArtifactNames/action.yaml	Added CodeCoverageArtifactsName output	Action contract
CheckForUpdates.HelperFunctions.ps1	Added MergeCoverage to PostProcess needs	Workflow update logic
RunPSScriptAnalyzer.ps1	Added TestRunner path suppressions for ported BC code patterns	CI check
_BuildALGoProject.yaml (both templates)	Added coverage artifact upload + BuildCodeCoverageSummary step	Workflow
CICD.yaml (both templates)	Added MergeCoverage job	Workflow
PullRequestHandler.yaml (both templates)	Added MergeCoverage job	Workflow

---

Settings

{
    "enableCodeCoverage": true,
    "codeCoverageSetup": {
        "trackingType": "PerRun",
        "produceCodeCoverageMap": "PerCodeunit",
        "excludeFilesPattern": ["*.PermissionSet.al"]
    }
}

Setting	Default	Description
enableCodeCoverage	false	Master switch. When false, no code coverage code executes.
trackingType	PerRun	Granularity: PerRun (one file per run), PerCodeunit (per codeunit), PerTest (per test function). Higher granularity = more overhead.
produceCodeCoverageMap	PerCodeunit	Map file granularity: Disabled, PerCodeunit, PerTest
excludeFilesPattern	[]	Glob patterns for AL files to exclude from coverage denominator

---

Test Coverage

Unit Tests (220 tests)

Test File	Module Tested	Tests
ALSourceParser.Test.ps1	AL source parsing + Find-ALSourceFolders	55
BCCoverageParser.Test.ps1	BC coverage data parsing	26
CoberturaFormatter.Test.ps1	Cobertura XML generation	33
CoberturaMerger.Test.ps1	Multi-file merge logic	24
CoverageProcessor.Test.ps1	End-to-end processing	29
CoverageReportGenerator.Test.ps1	Report generation	43
BuildCodeCoverageSummary.Test.ps1	Summary action	10
MergeCoverageSummaries.Test.ps1	Merge action	11
NewALTestRunnerOverride.Test.ps1	Override scriptblock (URL, params, results)	17
TestResultFormatter.Test.ps1	JUnit/XUnit formatting	12

What's NOT unit tested (and why)

The ported BC test runner internals (Run-AlTestsInternal, ClientContext, ClientSessionManager, TestFormHelpers, CoverageCollector) require a running BC container to test meaningfully. These are covered by BC's own test infrastructure and by the AL-Go E2E tests that run against real containers. Unit testing them would require mocking the entire BC client services stack, which provides little value for code that is already proven in production.

---

Risk Assessment

Area	Risk	Mitigation
Feature disabled	None	All code gated by enableCodeCoverage check
RunPipeline changes	Low	Override injection is additive; existing paths unchanged
Workflow changes	Low	MergeCoverage job no-ops when disabled; PostProcess dependency added
CalculateArtifactNames	None	Additive (new output only)
Ported test runner	Low	Known-good BC code; decomposed but not rewritten
CoverageProcessor (new)	Medium	Most complex new code; thoroughly tested (165 tests)
Custom override conflict	None	Detected and warned; user's override preserved

---

Review Priorities

High priority (new logic — review carefully):

1. CoverageProcessor/ — all 5 modules (core new feature)
2. RunPipeline.psm1 — override scriptblock
3. RunPipeline.ps1 — coverage integration points (lines 465-497, 762-835)
4. BuildCodeCoverageSummary/ and MergeCoverageSummaries/ — new actions

Medium priority (modified existing code):
5. ALTestRunner.psm1 — JUnit support addition
6. TestResultFormatter.psm1 — new result formatting
7. Workflow template changes (6 YAML files)
8. CalculateArtifactNames changes

Low priority (ported code — minimal changes):
9. Internal/ modules — decomposed from original, functionally identical
10. DLL files — identical to BC platform originals

[github-advanced-security]

PSScriptAnalyzer found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.