Skip to content

fix(pool): use stable NNTP provider IDs - #840

Draft
dclipca wants to merge 1 commit into
javi11:mainfrom
dclipca:security/redact-nntp-provider-identities
Draft

fix(pool): use stable NNTP provider IDs#840
dclipca wants to merge 1 commit into
javi11:mainfrom
dclipca:security/redact-nntp-provider-identities

Conversation

@dclipca

@dclipca dclipca commented Aug 23, 2026

Copy link
Copy Markdown
Contributor

Summary

nntppool PR #87 adds an optional stable Provider.Name with the existing host/account fallback for callers that do not provide one. This change validates unique non-empty provider IDs, sets Name: p.ID at the AltMount boundary, and uses the ID for pool lookup, metrics, speed tests, and propagated errors. Authentication usernames therefore do not enter external provider names while endpoint host context remains available where it is operationally useful.

Provider status no longer includes an authentication username. Frontend types and checked-in Swagger/OpenAPI artifacts are synchronized. Existing quota keys are migrated atomically from the legacy host/account form to stable IDs; ambiguous legacy keys are retained rather than credited to multiple IDs.

The companion nntppool change must merge and release before this PR is merge-ready. No contributor-fork replace is part of this branch.

Tests

  • nntppool PR Failed to get health records when release date is null #87: go test -race ./... and go vet ./... pass.
  • AltMount focused config/pool/database tests pass.
  • AltMount focused race tests for config/pool/database and API pass.
  • AltMount full go vet ./... passes.
  • AltMount full go test ./... reaches the complete suite; only the existing environment-sensitive internal/metadata directory-mtime and internal/updater writability tests fail.
  • Frontend build is not runnable in this checkout because frontend dependencies (tsc) are not installed.

@javi11

javi11 commented Aug 23, 2026

Copy link
Copy Markdown
Owner

Before landing this, I think we can avoid most of the complexity by fixing it at the source.

nntppool derives the provider key as host + "+" + Auth.Username (resolveProviderName), which is the only reason we need a redaction boundary here. Since we own nntppool, I'd rather add an optional Provider.Name field (falling back to the current derivation when empty) and set Name: p.ID in ToNNTPProvider(). NNTPPoolName() then just returns p.ID, and the key is safe, stable and unique by construction.

That lets us drop provider_safety.go + its test, the regex fallback, the #1/#2 ordinals and all the call-site rewrites. The PR shrinks to: the upstream field, two small changes in ToNNTPProvider/NNTPPoolName, and deleting ProviderStatus.Username — nothing in frontend/src reads it, so we don't need the Username: "" placeholder either.

It also fixes the leak we're leaving in place today: quota_used:host:port+username in system_stats becomes quota_used:<provider-id>.

The one real cost is quota state: injectQuotaState looks up quota_used:<name>, so the keys change and quota windows restart once. Either we accept that, or add a small one-time migration that rewrites any quota_used:/quota_reset_at: key still shaped like host:port[+user] to the matching provider ID.

Two things worth flagging regardless of which way we go:

  1. The #1/#2 ordinals are built from a different name set at each boundary — enabled config providers in handleGetPoolMetrics, pool stats in safeProviderLabel, stats+name in RemoveProvider — so the same provider can be x#1 in one place and x#2 in another. In handleGetPoolMetrics the ps.Name == labels.Label(p.NNTPPoolName()) match compares ordinals derived from two different sets, so if a provider is disabled but still in the pool the ordinals shift and metrics attach to the wrong provider row. safeProviderInt64Map then merges counters by that label. We already have a stable, non-sensitive identity for this (the config ID, via SetProviderIDs).

  2. GetPool() now allocates a redactingClient and calls client.Stats() on every call, and it's called per-file in the importer and per-batch in validation. Better to cache one wrapper in the manager and rebuild it in SetProviders/AddProvider/RemoveProvider.

If we do keep the boundary, the redactingClient chokepoint is the right shape — that part I'd keep, along with sanitizedProviderError preserving Unwrap.

@dclipca

dclipca commented Aug 23, 2026

Copy link
Copy Markdown
Contributor Author

Follow-up to the source-level design: companion nntppool PR #87 adds the optional stable Provider.Name field with legacy fallback. AltMount will set Name to the validated provider ID and use that ID for pool metrics/errors; the canonical dependency must merge/release before this PR is merge-ready. Quota state migration and API contract updates are included here.

@dclipca
dclipca force-pushed the security/redact-nntp-provider-identities branch from d9ebc40 to 7fea717 Compare August 23, 2026 21:17
@dclipca dclipca changed the title fix(pool): redact NNTP provider account labels at boundaries fix(pool): use stable NNTP provider IDs Aug 23, 2026
@dclipca

dclipca commented Aug 23, 2026

Copy link
Copy Markdown
Contributor Author

Update: tip 7fea717 passes the local full suite with the review-only companion module, focused race/vet, and API/docs checks. Canonical CI build-docs passes. Canonical test is expectedly blocked until nntppool PR #87 is merged/released: it currently compiles against v4.15.0 and reports unknown field Name in nntppool.Provider at the two source-boundary uses. No contributor-fork replace is present in this PR.

@dclipca
dclipca marked this pull request as draft August 23, 2026 21:19
@dclipca
dclipca force-pushed the security/redact-nntp-provider-identities branch from 7fea717 to 36e5503 Compare August 23, 2026 21:30
@dclipca

dclipca commented Aug 23, 2026

Copy link
Copy Markdown
Contributor Author

Follow-up at 36e5503: migration failures now log only a fixed repository classification; a capture regression verifies raw repository text cannot reach logs. Provider IDs are documented and validated as stable public non-secret identifiers: unique, exact-trim, and graphic-only (including bidi/line-separator protection). Focused API/config/pool/database tests, race tests, and vet pass locally against the review-only nntppool #87 module; the replace is not committed.

@javi11

javi11 commented Aug 26, 2026

Copy link
Copy Markdown
Owner

I already published the 4.17.0 can you change it?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants