Skip to content

test(e2e): report what the gateway refused - #14

Merged
arloliu merged 1 commit into
mainfrom
test/console-auth-failure-report
Aug 30, 2026
Merged

test(e2e): report what the gateway refused#14
arloliu merged 1 commit into
mainfrom
test/console-auth-failure-report

Conversation

@arloliu

@arloliu arloliu commented Aug 30, 2026

Copy link
Copy Markdown
Owner

What a console failure could not say

A console scenario that fails on a control it cannot find has usually left the
console for the login. The page goes there because request()
(internal/ui/static/app.js:401) got a 401 and called signIn(), which sets
phase to navigating and empties .panels.

Which fetch was refused, and why, decided nothing the scenario could show.
The merged chooseOption change names where the page went; this names why
it went there.

The counter already existed

profgate_auth_failures_total{mode,reason}
(internal/metrics/prometheus.go:103) counts every request the auth layer
refused. The reason vocabulary is auth.Reasons() (internal/auth/auth.go:76),
so session (the cookie would not open or had expired), csrf
(Sec-Fetch-Site), and the callback reasons are told apart at the point of
failure.

No test read it before this.

The change

Both console scenarios forward the ops port and log every non-zero counter —
only when the scenario failed. A passing run logs nothing and its output is
unchanged.

The port-forward is registered for cleanup before the scrape is, so the scrape
runs first and still has a forward. h.forward binds its lifetime to stopCh
rather than to the caller's context, so it survives into cleanup; the scrape
carries a context of its own because the test's is cancelled by then.

The baseline this establishes

Three passing runs, scraped before and after the second load:

before the second load    profgate_auth_failures_total{mode="oidc",reason="missing"} 1
after choosing a profile  profgate_auth_failures_total{mode="oidc",reason="missing"} 1

One refusal, reason missing, counted during the first load — which
carries no session and is meant to be refused. The count across the second
load is zero.
So an extra refusal there is the anomaly, and its reason is the
answer to why console-oidc fails on the lane.

Proof

case outcome
scenario navigates to the login before choosing FAIL, and the counter is logged
unmodified both console scenarios PASS, nothing logged

The full suite on the current lane logged no counter at all across every
scenario.

Validation

mise run lint (0 issues), mise run test, mise run check, and semlf --base origin/main pass. mise run test:e2e ran green on the current lane
in 696.9s.

A console scenario that fails on a control it cannot find has usually left the console for the login,
and the page goes there because something answered one of its fetches 401.
Which fetch, and why, decided nothing the scenario could show.

The gateway already counts it.
profgate_auth_failures_total carries the mode and the reason,
and the vocabulary is the one auth.Reasons names,
so session, csrf, and the callback reasons are told apart at the point of failure.
Nothing read the counter before this.

Both console scenarios now forward the ops port and log every non-zero counter,
but only when the scenario failed:
a passing run says nothing and keeps its output as it was.

Three passing runs put the baseline at one refusal, reason missing,
counted during the first load, which carries no session and is meant to be refused.
The count across the second load is zero,
so an extra refusal there is the anomaly and its reason is the answer.

Proven both ways against a scenario that navigates to the login before choosing:
the failing run logged the counter, and the passing run logged nothing.
@arloliu
arloliu merged commit 43ca6ba into main Aug 30, 2026
6 checks passed
@arloliu
arloliu deleted the test/console-auth-failure-report branch August 30, 2026 07:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant