feat: adapt to gateway-hostname DIDs, add dual-VE DNS wiring - #3
Merged
Conversation
The platform now advertises itself under its gateway hostname — DSP callback, credential service, issuance endpoint and did:web identifiers all resolve there rather than to in-cluster Service FQDNs. The runtimes dereference those URLs themselves (the control plane resolves the issuer DID to verify credential signatures, and calls the CredentialService on every DSP message), so a `*.localhost` name resolving to the pod's own loopback breaks every lookup. setup-did-dns.sh adds CoreDNS rewrites pointing each HTTPRoute hostname at the Traefik Service, then verifies both DNS and an end-to-end did:web resolution. Only the cluster name is a parameter: the DNS domain, Traefik Service and hostname list are read off the cluster, so the rules cannot drift from what is deployed. That matters most for the DNS domain, since each VE has its own and a target under `cluster.local` NXDOMAINs on a VE built with a custom one. It is idempotent (rules live between marker comments) and install-ve.sh now runs it after the platform install, so a failure stops the install instead of resurfacing later as a credential-verification error. The probe image is deliberately glibc and deliberately not curl: musl (Alpine, hence curlimages/curl) and curl itself each resolve *.localhost to loopback internally without querying DNS, so either would fail whether or not the rewrite is correct. install-ve.sh also pins cx-profile's issuer.did, which must equal the DID the platform minted, and both scripts now mint participant DIDs under identity.<host> to match the platform's IdentityHub did:web hostname — the previous in-cluster template would have overridden the chart default straight back to a DID that no longer resolves. Peer rewrites for the dual-VE setup are not covered yet: each cluster additionally needs the peer's hostnames pointed at traefik.traefik.svc.<peer-dns-domain>. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Adapts the multi-VE flow to the platform advertising itself under gateway hostnames (ve1.localhost, issuer.ve1.localhost, identity.ve1.localhost) instead of in-cluster Service FQDNs. Verified end to end: fresh dual-VE install, onboarding per VE, connect, cross-VE DSP exchange to FINALIZED/STARTED with payload download, then the whole e2e again on the same clusters to prove idempotency. setup-did-dns.sh gains a peer mode (--peer <cluster>): it makes one VE resolve the other's gateway hostnames, under which everything the peer advertises lives. A CoreDNS `rewrite` cannot do this — a rewritten name does not re-enter server-block selection, so it would bypass the peer-zone forwarding and NXDOMAIN — hence a dedicated multi-zone server block forwards the peer's hostnames themselves to the peer's kube-dns, whose own rewrite rules answer with the peer's Traefik ClusterIP, tracking it automatically. Hostnames the local cluster also serves (the telemetry hosts, identical in every VE) are excluded, since a dedicated server block wins zone selection and would hijack the local name. Own and per-peer Corefile blocks are delimited by exact-match markers so they replace only themselves. Peer mode verifies its result: DNS from a probe pod against the peer's Traefik ClusterIP, plus an end-to-end fetch of the peer issuer's DID document asserting the document id equals the DID. connect-ves.sh runs peer mode for both directions after routes + zone forwarding, and trusts the peer issuer under its real DID (did:web:issuer.<peer-host>:issuer) instead of the defunct in-cluster form. The old smoke test is gone: it fetched the in-cluster Service URL nothing advertises anymore (a false positive), with an image that cannot resolve *.localhost through DNS at all. dsp-tests.sh hands the consumer the provider's advertised DSP endpoint (http://<prov-host>/api/dsp/…) instead of the in-cluster URL, dropping the now-unused --pd/--cd flags. mgmt() retries 502/503/504: those mean the gateway never reached the backend, a real window right after connect-ves.sh's controlplane restarts while Traefik's endpoint view lags the rollout — previously masked by the smoke test's incidental delay, and observed (and absorbed by the retry) during verification. The app's own fallback DID template (@value default and application.yaml) now matches the chart default, did:web:identity.cxve.localhost: — the packaged application.yaml always defines the property, so the stale yaml value would have kept winning over the corrected Java default. install-ve.sh/README: the VE's identity domain is now the -H hostname (DIDs embed identity.<host> / issuer.<host>); distinct -d DNS domains remain required, but for cross-cluster zone forwarding, not identity. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
e2e.sh and connect-ves.sh already defaulted to charts/cx-ve; install-ve.sh was the one place still defaulting to the published OCI chart, so a direct run deployed the published chart around the freshly built working-tree image. --version stays: helm applies it to remote refs and ignores it for a chart directory. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Only resolved from inside the clusters today, but the step's purpose is deterministic resolution — and a future host-side check against a DID URL would otherwise NXDOMAIN in getent-based tools while curl silently short-circuits *.localhost, a confusing split. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
wolf4ood
approved these changes
Jul 28, 2026
… window A one-shot check right after `rollout status` races the new Corefile becoming authoritative: lameduck pods still answer with the old config and the kubelet can mount a stale ConfigMap volume, during which *.localhost queries fall through to the host resolver and get ::1. Both probes now retry for up to 3 minutes before judging. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The platform now advertises its DSP/DCP endpoints and did:web identifiers under gateway hostnames (
ve1.localhost,issuer.ve1.localhost,identity.ve1.localhost) instead of in-cluster Service FQDNs. This adapts cx-ve accordingly.These gateway hostnames are also used inside the cluster, hence a change of the CoreDNS configuration is necessary (
scripts/setup-did-dns.sh)Changes
scripts/setup-did-dns.sh(new): makes the gateway hostnames resolvable from inside a cluster — the runtimes dereference the advertised URLs and DIDs themselves. Own mode adds CoreDNS rewrites to the cluster's Traefik; peer mode (--peer) forwards the peer VE's hostnames to the peer's kube-dns for the dual-VE setup. Idempotent, self-discovering, with built-in verification (DNS + end-to-end DID document fetch).connect-ves.sh: runs peer mode both ways; trusts the peer issuer under its real DID (did:web:issuer.<peer-host>:issuer); drops the now-false-positive smoke test.install-ve.sh: runs setup-did-dns.sh after the platform install, pins cx-profile'sissuer.did, defaults to the working-tree obapi chart.dsp-tests.sh: uses the provider's advertised DSP endpoint (http://<host>/api/dsp/…); retries 502/503/504 (gateway endpoint staleness right after the connect-time controlplane restarts).did:web:identity.<host>:(chart, scripts, app defaults).🤖 Generated with Claude Code