Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/bug.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ body:
id: version
attributes:
label: LayoutScope version
placeholder: 0.3.0
placeholder: 0.4.0
validations:
required: true
- type: dropdown
Expand Down
42 changes: 42 additions & 0 deletions .github/workflows/action-smoke.yml
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,38 @@ jobs:
if (report.options.paths?.length !== 1 || report.options.paths[0] !== "/") process.exit(1);
if (report.options.viewports?.length !== 1 || report.options.viewports[0]?.name !== "mobile") process.exit(1);
if (report.options.waitMs !== 0) process.exit(1);
if (report.comparison !== undefined) process.exit(1);
NODE
- id: layoutscope-compare
uses: ./layoutscope-action
with:
target: layoutscope-action/demo/broken/index.html
config: layoutscope-action/demo/broken/layoutscope.config.json
compare: ${{ steps.layoutscope.outputs.json-report }}
output: .layoutscope-compare
fail-on: error
artifact-name: layoutscope-compare-report
- name: Verify baseline comparison outputs
env:
BASELINE_REPORT: ${{ steps.layoutscope.outputs.json-report }}
HTML_REPORT: ${{ steps.layoutscope-compare.outputs.html-report }}
JSON_REPORT: ${{ steps.layoutscope-compare.outputs.json-report }}
SARIF_REPORT: ${{ steps.layoutscope-compare.outputs.sarif-report }}
run: |
test "$HTML_REPORT" = "$(realpath -m .layoutscope-compare/report.html)"
test "$JSON_REPORT" = "$(realpath -m .layoutscope-compare/report.json)"
test "$SARIF_REPORT" = "$(realpath -m .layoutscope-compare/report.sarif.json)"
test -s "$HTML_REPORT" && test -s "$JSON_REPORT" && test -s "$SARIF_REPORT"
node --input-type=commonjs <<'NODE'
const report = require(process.env.JSON_REPORT);
if (report.summary.error < 1) process.exit(1);
if (!report.comparison) process.exit(1);
const counts = report.comparison.counts;
if (counts?.new !== 0 || report.comparison.new?.length !== 0) process.exit(1);
if (counts?.resolved !== 0 || report.comparison.resolved?.length !== 0) process.exit(1);
if (counts?.existing !== report.summary.total) process.exit(1);
if (report.comparison.existing?.length !== counts.existing) process.exit(1);
if (report.comparison.baseline?.source !== process.env.BASELINE_REPORT) process.exit(1);
NODE
- uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
Expand All @@ -61,6 +93,16 @@ jobs:
test -s .artifact-download/report.json
test -s .artifact-download/report.sarif.json
test "$(find .artifact-download/screenshots -type f -name '*.png' | wc -l)" -ge 2
- uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: layoutscope-compare-report
path: .compare-artifact-download
- name: Verify comparison artifact
run: |
test -s .compare-artifact-download/report.html
test -s .compare-artifact-download/report.json
test -s .compare-artifact-download/report.sarif.json
test "$(find .compare-artifact-download/screenshots -type f -name '*.png' | wc -l)" -ge 2

failure-threshold:
runs-on: ubuntu-latest
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/release-smoke.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ jobs:
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- id: layoutscope
uses: LyraZeta/layoutscope@v0.3.0
uses: LyraZeta/layoutscope@v0.4.0
with:
target: demo/broken/index.html
config: demo/broken/layoutscope.config.json
Expand All @@ -28,6 +28,6 @@ jobs:
test -s "$SARIF_REPORT"
node --input-type=commonjs <<'NODE'
const report = require(process.env.JSON_REPORT);
if (report.tool?.version !== "0.3.0") process.exit(1);
if (report.tool?.version !== "0.4.0") process.exit(1);
if (report.summary?.suppressed !== 1 || report.suppressions?.total !== 1) process.exit(1);
NODE
18 changes: 18 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,24 @@

All notable changes to LayoutScope are documented here.

## 0.4.0 - 2026-08-13

### Added

- Optional `--compare <report.json>` fingerprint comparison for pull-request visual gates without pixel baselines.
- `new`, `existing`, and `resolved` finding groups in JSON, a comparison summary in the interactive HTML report, and comparison metadata in SARIF.
- A matching GitHub Action `compare` input plus a recipe for downloading the latest successful main-branch report.

### Changed

- In comparison mode, terminal output and `--fail-on` consider only new findings. The complete current scan remains available in every report for audit and debugging.
- Finding comparison uses route, full viewport identity, rule ID, and fingerprint, with multiset matching for repeated findings.

### Safety

- Baseline reports are parsed and structurally validated before config loading, browser launch, or report output.
- Comparison excludes suppression audit samples and never writes resolved baseline findings into the current page results.

## 0.3.0 - 2026-08-13

### Added
Expand Down
66 changes: 51 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,28 +27,28 @@ LayoutScope needs no baseline. It inspects the live DOM, open Shadow DOM, comput
Try LayoutScope on a public page without setting up a local project:

```bash
npx --yes github:LyraZeta/layoutscope#v0.3.0 \
npx --yes github:LyraZeta/layoutscope#v0.4.0 \
'https://marmelab.com/react-admin-demo/#/login' \
--viewport mobile --wait 1500 --fail-on never
```

Scan your app:

```bash
npx --yes github:LyraZeta/layoutscope#v0.3.0 http://localhost:3000
npx --yes github:LyraZeta/layoutscope#v0.4.0 http://localhost:3000
```

Scan a local HTML file:

```bash
npx --yes github:LyraZeta/layoutscope#v0.3.0 ./dist/index.html
npx --yes github:LyraZeta/layoutscope#v0.4.0 ./dist/index.html
```

After your app server is listening in the same GitHub Actions job, run the
LayoutScope step against its loopback URL:

```yaml
- uses: LyraZeta/layoutscope@v0.3.0
- uses: LyraZeta/layoutscope@v0.4.0
with:
target: http://127.0.0.1:3000
```
Expand All @@ -66,7 +66,7 @@ For complete copy-paste workflows, see the
Use specific routes and viewports:

```bash
npx --yes github:LyraZeta/layoutscope#v0.3.0 http://localhost:3000 \
npx --yes github:LyraZeta/layoutscope#v0.4.0 http://localhost:3000 \
--path / --path /pricing --path /dashboard \
--viewport desktop --viewport mobile --viewport 768x1024:tablet
```
Expand All @@ -83,10 +83,10 @@ screenshots/ Raw and annotated screenshots
LayoutScope uses an installed Chrome or Edge automatically when available. Otherwise install the Playwright browser once after cloning:

```bash
npx --yes --package github:LyraZeta/layoutscope#v0.3.0 playwright-core install chromium
npx --yes --package github:LyraZeta/layoutscope#v0.4.0 playwright-core install chromium
```

The default failure threshold is `error`, so finding a real problem returns a non-zero exit code. Use `--fail-on never` while exploring.
The default failure threshold is `error`, so finding a real problem returns a non-zero exit code. Use `--fail-on never` while exploring. LayoutScope uses exit code `1` for findings at the configured threshold and `2` for configuration, baseline, browser, or scan errors.

## What It Finds

Expand All @@ -110,30 +110,30 @@ Every rule is deterministic and runs locally. LayoutScope understands modern CSS
LayoutScope exits non-zero when an error is found by default:

```bash
npx --yes github:LyraZeta/layoutscope#v0.3.0 http://127.0.0.1:3000 --no-open --fail-on error
npx --yes github:LyraZeta/layoutscope#v0.4.0 http://127.0.0.1:3000 --no-open --fail-on error
```

Choose a threshold:

```bash
npx --yes github:LyraZeta/layoutscope#v0.3.0 ./site --fail-on critical
npx --yes github:LyraZeta/layoutscope#v0.3.0 ./site --fail-on warning
npx --yes github:LyraZeta/layoutscope#v0.3.0 ./site --fail-on never
npx --yes github:LyraZeta/layoutscope#v0.4.0 ./site --fail-on critical
npx --yes github:LyraZeta/layoutscope#v0.4.0 ./site --fail-on warning
npx --yes github:LyraZeta/layoutscope#v0.4.0 ./site --fail-on never
```

Authenticated page:

```bash
npx --yes github:LyraZeta/layoutscope#v0.3.0 https://app.example.com/dashboard \
npx --yes github:LyraZeta/layoutscope#v0.4.0 https://app.example.com/dashboard \
--storage-state ./playwright/.auth/user.json \
--ready '[data-dashboard-ready]'
```

Manual GitHub Actions steps:

```yaml
- run: npx --yes --package github:LyraZeta/layoutscope#v0.3.0 playwright-core install --with-deps chromium
- run: npx --yes github:LyraZeta/layoutscope#v0.3.0 http://127.0.0.1:3000 --no-open
- run: npx --yes --package github:LyraZeta/layoutscope#v0.4.0 playwright-core install --with-deps chromium
- run: npx --yes github:LyraZeta/layoutscope#v0.4.0 http://127.0.0.1:3000 --no-open
- uses: actions/upload-artifact@v7
if: always()
with:
Expand All @@ -143,6 +143,41 @@ Manual GitHub Actions steps:

Upload `report.sarif.json` with `github/codeql-action/upload-sarif` to surface findings in Code Scanning.

### Block only new findings

Adopt LayoutScope without fixing the entire existing backlog first. Save a
normal JSON report from the main branch, then compare the pull request against
that report:

```bash
# Main branch inventory: detection itself still needs no reference screenshot.
npx --yes github:LyraZeta/layoutscope#v0.4.0 http://127.0.0.1:3000 \
--output .layoutscope-main --no-open --fail-on never

# Pull request gate: fail only when a new finding meets the threshold.
npx --yes github:LyraZeta/layoutscope#v0.4.0 http://127.0.0.1:3000 \
--compare .layoutscope-main/report.json --output .layoutscope-pr --no-open
```

Comparison mode matches deterministic findings by route, viewport, rule, and
fingerprint. Terminal output focuses on new findings, while `report.json`
retains the complete current scan plus `new`, `existing`, and `resolved`
groups. The HTML and SARIF reports include the same comparison counts. Use the
same paths, viewports, and config on both runs for a meaningful comparison.

The Action exposes the same option:

```yaml
- uses: LyraZeta/layoutscope@v0.4.0
with:
target: http://127.0.0.1:3000
compare: .layoutscope-main/report.json
```

See the [pull-request baseline recipe](docs/recipes.md#pull-request-baseline-gate)
for a complete workflow that stores the latest successful main-branch report
as a GitHub Actions artifact.

### Suppress reviewed findings

Create `layoutscope.config.ts`, `.js`, or `.json` when a deterministic finding
Expand Down Expand Up @@ -196,6 +231,7 @@ layoutscope <target> [options]

-c, --config <path> Config file; auto-detects layoutscope.config.*
--no-auto-config Disable automatic config discovery
--compare <report.json> Compare findings with an earlier JSON report
-o, --output <directory> Report directory (default: .layoutscope)
-v, --viewport <viewport> desktop, mobile, tablet, wide, or WIDTHxHEIGHT
-p, --path <path> Route to scan; repeatable
Expand Down Expand Up @@ -249,7 +285,7 @@ The demo intentionally contains responsive overflow, clipped content, a collisio

## Roadmap

- GitHub Action PR summaries and SARIF upload examples
- Native GitHub pull-request summaries and annotations
- Inline suppression comments for fixture-heavy projects
- Multi-page discovery from a sitemap
- Fixed/sticky occlusion and focus-ring clipping rules
Expand Down
6 changes: 6 additions & 0 deletions action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,10 @@ inputs:
description: "CI threshold. Omit to use the config or CLI default."
required: false
default: ""
compare:
description: Existing LayoutScope report.json to compare against. Omit to report all current findings.
required: false
default: ""
upload-artifact:
description: Upload the report as a workflow artifact.
required: false
Expand Down Expand Up @@ -103,6 +107,7 @@ runs:
INPUT_MAX_FINDINGS: ${{ inputs['max-findings'] }}
INPUT_INCLUDE_INFO: ${{ inputs['include-info'] }}
INPUT_FAIL_ON: ${{ inputs['fail-on'] }}
INPUT_COMPARE: ${{ inputs.compare }}
run: |
set -euo pipefail
args=("$INPUT_TARGET" --output "$INPUT_OUTPUT" --no-open --no-auto-config)
Expand All @@ -122,6 +127,7 @@ runs:
[[ -n "$INPUT_EXECUTABLE_PATH" ]] && args+=(--executable-path "$INPUT_EXECUTABLE_PATH")
[[ -n "$INPUT_MAX_FINDINGS" ]] && args+=(--max-findings "$INPUT_MAX_FINDINGS")
[[ -n "$INPUT_FAIL_ON" ]] && args+=(--fail-on "$INPUT_FAIL_ON")
[[ -n "$INPUT_COMPARE" ]] && args+=(--compare "$INPUT_COMPARE")
[[ "$INPUT_INCLUDE_INFO" == "true" ]] && args+=(--include-info)
node "$GITHUB_ACTION_PATH/dist/cli.js" "${args[@]}"
- name: Expose report paths
Expand Down
Loading
Loading