feat(spectre): implement Spectre dCP operator (Phases 0-D) - #506
Open
xCatalitY wants to merge 56 commits into
Open
feat(spectre): implement Spectre dCP operator (Phases 0-D)#506xCatalitY wants to merge 56 commits into
xCatalitY wants to merge 56 commits into
Conversation
|
You are seeing this message because GitHub Code Scanning has recently been set up for this repository, or this pull request contains the workflow file for the Code Scanning tool. What Enabling Code Scanning Means:
For more information about GitHub Code Scanning, check out the documentation. |
…ffold listener handler
The kubebuilder scaffold doubled the API group to spectre.ei.telekom.de.cp.ei.telekom.de because the --group flag was passed as "spectre.ei.telekom.de" instead of just "spectre". This corrects the group across all source and config files: - groupversion_info.go: fix +groupName marker and GroupVersion var - PROJECT: fix group field from "spectre.ei.telekom.de" to "spectre" - listener_types.go: align with controller-runtime scheme.Builder - controller RBAC markers: fix apiGroups - config/rbac/*.yaml: fix apiGroups - config/crd/kustomization.yaml: fix CRD filename reference - config/samples: rename file and fix apiVersion - Makefile: fix boilerplate paths to use ../hack/ (repo root)
Add local replace directives for all monorepo dependencies (common, common-server, admin/api, application/api, approval/api, gateway/api, identity/api, pubsub/api, secret-manager, event/api) matching the pattern used by the event module. Also bumps go directive to 1.26.5 as required by the common module.
Replace spectre-system with controlplane-system in kustomization and e2e tests to match the shared controlplane namespace convention used by all other domain modules (event, gateway, etc.).
…Application CRDs Replace the scaffolded placeholder ListenerSpec (with single Foo field) with the full Listener and SpectreApplication CRD types from the design doc. Both types implement the common types.Object interface (GetConditions, SetCondition) and have compile-time assertions. Changes: - ListenerSpec: consumer/provider TypedObjectRefs, optional ApiListener and EventListener with filter support - ListenerStatus: conditions, routeListener, eventSubscriptions, providerApproval, consumerApproval refs - SpectreApplicationSpec: application ref, deliveryType enum with CEL XValidation for callback field - SpectreApplicationStatus: conditions, id, publisher, subscriber, listenerRoute, proxyRoute refs - schema.go: RegisterSchemesOrDie for all cross-domain API types - SpectreApplication controller + handler stubs - cmd/main.go: wires both controllers via RegisterSchemesOrDie - Test updated to initialize controller properly with required spec fields
…ion + add sample CR
Implement handler utility package for the Spectre operator:
- GetListeningZone: ports legacy ListenerUtil preference logic
(listener zone > provider zone > consumer zone > blocked)
- GetEventConfig: resolves EventConfig for a zone via field index
- Naming helpers: MakePublisherName, MakeSubscriberName,
MakeRouteListenerName, MakeBridgeSubscriberId,
BuildListenerEventType, BuildBridgeCallbackURL
- Constants: PublisherID ("gateway"), GenericEventType
27 Ginkgo specs with 96.8% coverage.
… + Subscriber + SSE Route)
…stener + bridge Subscribers)
Bug 1: RouteListener Spec.Route referenced itself (routeListenerName) instead of the actual gateway Route CR. Added findRouteByPath helper that resolves the Route by matching Spec.Paths against apiBasePath, and returns a BlockedError if no Route exists yet. Bug 2: GatewayClient credentials (ClientId, Issuer) were not propagated to the JumperConfig RouteListenerEntry. Extended the struct and updated the Apply() method to copy them from the RouteListener spec. Bug 3: RouteListener listing was placed outside the !PassThrough guard, causing unnecessary listing for passthrough routes that skip security features entirely. Moved inside the guard block. Bug 4: Route controller did not watch RouteListener changes, so RouteListener creation/update would not trigger Route re-reconciliation. Added Watches() with mapRouteListenerToRoute mapping function.
…t-least-one listener type
Add Spectre listener support to the rover-server REST API: - Define RoverListener, ListenerFilter, and ListenerSubscription schemas in the OpenAPI spec - Regenerate server types via oapi-codegen - Add bidirectional mapper functions (API <-> CRD) for listeners and listenerSubscription following existing exposure/subscription patterns - Add comprehensive Ginkgo tests for both mapping directions
…, nil guard) - Use content-based listener naming (consumer+path/eventType) instead of fragile array index, preventing orphaned CRs on reorder/removal - Zero SpectreApplications/SpectreListeners status slices before early return so removed listeners don't leave stale refs - Only set callback on SpectreApplication when deliveryType is "callback", preventing XValidation rejection for SSE subscriptions - Add nil guard for rover.Status.Application returning BlockedError - Use two-value type assertion in rover-ctl test to avoid panics
…ed Namespace resource
…e E2E test suite Bugs fixed: - spectre/cmd/main.go: add setupIndexes() for EventConfig zone field index and owner indexes (ApprovalRequest, Subscriber, Publisher, Route, RouteListener) - spectre/config/manager/manager.yaml: fix image ref (controller:latest → ghcr.io/telekom/controlplane/spectre:stable) and remove command: [/manager] (ko uses entrypoint /ko-app/cmd) - spectre RBAC: add eventstores to pubsub resource permissions - rover/cmd/main.go: add spectrev1.AddToScheme (feature-gated) - rover RBAC: add spectre CRD permissions (spectreapplications, listeners) - spectre handler test: fix mock for resolveSpectreApplication (Get not List) New: - hack/spectre-e2e.sh: 2365-line E2E test script with 13 scenarios covering SSE/callback delivery, multi-listener, cascade delete, rover-driven creation, spec field verification, approval denied/cross-team, and edge cases - hack/local-setup.sh: add spectre to ALL_CONTROLLERS build list - install/overlays/local: spectre image tag + FEATURE_SPECTRE_ENABLED on rover
…rs present Jumper routes traffic through /proxy (standard) or /listener (Spectre capture) based on the Kong service path. The RouteListener feature was only populating the jumper_config header but never switching the upstream from /proxy to /listener, causing Spectre filters to never fire. Add LocalhostListenerUrl constant and call builder.SetUpstream in the RouteListenerFeature.Apply to route traffic through jumper's listener route which includes SpectreRequestFilter and SpectreResponseFilter.
Adds build, test, lint, image build, and vulnerability scan for the spectre module. Uses the same reusable-go-ci workflow as all other domain operators.
The envtest integration suite registers 8 CRD domains and runs 4 multi-step specs. On shared GitHub Actions runners this exceeds Go's default 10m timeout. Bump to 15m to give headroom.
The envtest integration suite (8 CRD domains, 4 multi-step specs) exceeds GitHub Actions runner capacity even with 15m timeout. Split into: - `make test`: handler + util unit tests (fast, CI) - `make test-integration`: envtest controller tests (local/nightly) Lower coverage threshold to 40% (unit tests cover handler 80.5% + util 96.8%, but excluding controller brings total to 42.8%).
CI's test-reporter step expects ginkgo-junit.xml. Without it the "Publish test report" step fails and blocks the image build.
The kubebuilder scaffold default of 10m is below the minimum enforced by the DOT cluster LimitRange, so the pod could never be created (FailedCreate: minimum cpu usage per Container is 30m). Align with the other controllers, which all request 30m.
The integration suite used record.NewFakeRecorder(100) with nothing draining the channel. ControllerImpl.Reconcile emits one event per pass and reconcileUntilReady polls every 250ms, so events accumulated until the 101st blocked forever on a plain channel send. The suite did not run slowly, it deadlocked: 12 Eventually blocks at 15s cap the arithmetic worst case at 3 minutes, yet CI timed out at both 10m and 15m, and the hang reproduced locally with goroutine 1 parked in [chan receive]. Drain the recorder in a goroutine. The suite now completes in ~8s. Un-hanging it exposed three stale fixtures the deadlock had masked: ListenerSpec.Application was added in c985d68 but never set, so Listeners blocked on SpectreApplication "/" not found, and one fixture additionally failed CRD admission by omitting ApiListener. With the suite green, drop the grep -v /controller exclusion, adopt the sibling TEST_PACKAGES/COVER_PACKAGES scoping, and raise the coverage threshold from 40 to 70 (now at ~74%).
…nsumer gate ensureApprovals built two ApprovalBuilders on the same Listener. Both share the one JanitorClient the framework injects per reconcile, and Build() ends with Cleanup(ApprovalRequestList, OwnedBy(owner)), which deletes every owner-owned ApprovalRequest absent from the client's tracked state and then wipes that state. So the second builder deleted the first one's request on every pass and the roles inverted on the next, meaning both approvals could never be Granted together and no Listener ever provisioned. Both gates also collapsed onto a single Approval CR, since ApprovalName derives from owner kind and name only. The consumer gate was a no-op: listenerTeam is consumerApp.Spec.Team, so computeStrategy(consumerTeam, consumerTeam) always returned Auto and the request was auto-granted. It existed only to collide with the provider gate. The provider owns the API whose traffic is captured, so provider consent is the security control; keep that gate alone. Also replace the "!= Granted" catch-all with an explicit switch that errors on an unknown builder result rather than waiting indefinitely, and drop ListenerStatus.ConsumerApproval, which can no longer be populated. The cross-team spec now grants via grantApprovalsForListener, which sets ApprovedRequest the way the real approval controller does. The previous fixture omitted it and so exercised a permissive path production never takes.
…cation lists Neither List type declared GetItems, so the janitor's cast in cleanupStateUnstructured failed with "object is not a valid list". rover registers both types in janitor state whenever the spectre feature gate is on, even with no listeners declared, and CleanupAll returns on the first error while iterating a Go map. So with the gate enabled, cleanup of every other Rover-owned type -- ApiExposure, ApiSubscription, EventSubscription, PermissionSet -- was skipped non-deterministically and every Rover was stuck NotReady. Mirrors gateway's RouteListenerList. The test asserts the interface is satisfied so this cannot regress silently at runtime.
…rusting owner refs Two comments claimed children cascade via Kubernetes owner references and SpectreApplication.Delete was a bare return nil on that basis, but no SetControllerReference call existed anywhere in the module and none could have worked: parents live in the team namespace while children are created in the zone namespace, and Kubernetes ignores cross-namespace owner refs. Every child leaked on delete, so an orphaned RouteListener kept capturing traffic and orphaned bridge Subscribers kept receiving payloads after the authorizing CR was gone. Delete from the refs already recorded in status, following the eventexposure handler. publisherNamespace prefers an existing child ref's namespace so cleanup still works once the referenced Applications are gone. Also correct the shared generic Publisher ref-count, which is one object per zone shared by all teams: list Listeners cluster-wide rather than scoped to the deleted Listener's namespace, exclude self by UID rather than name, skip Listeners already terminating, and tolerate IsNotFound on the delete so a repeat does not block finalizer removal. Gate Ready on AnyChanged before AllReady in both handlers. AllReady only turns false once a child reports Ready=False, and a freshly created child has no conditions, so the first reconcile reported Ready before anything was confirmed. Adds the paired Processing/DoneProcessing conditions the sibling domains set.
…election findRouteByPath matched the raw apiBasePath against Route.Spec.Paths, but those hold the preset-joined path (path.Join(preset.BasePath, apiBasePath)). The comparison only succeeded in a zone whose gateway preset basePath is "/" and silently found nothing otherwise, blocking the Listener with a misleading "no Route found". Fetch the Route by its derived name instead, mirroring the api domain's own MakeRouteName. That is also deterministic when several Routes share a path, which primary/secondary/proxy do by design. GetListeningZone was called with consumerZone as both the listener and the consumer argument, so the second SupportsZone check was trivially true (EventConfig.SupportsZone returns true for its own zone). The provider fallback was unreachable and the mesh restriction was bypassed: a provider zone outside the consumer's mesh still yielded the consumer zone. Reduce the signature to (providerZone, consumerZone) and state in the doc comment that the consumer zone is the listener zone, which is what the handler already assumes for the approval requester. PROJECT had version: "3" inside the resources list, which made the file unparseable and broke every kubebuilder invocation in the module, including kubebuilder version, since config loads before command dispatch.
…yClient 2ce4d47 added ClientId and Issuer inside RouteListenerEntry to propagate the gateway client. That is the wrong position in the wire contract: jumper's RouteListener model declares exactly issue and serviceOwner, and reads credentials from a separate top-level gatewayClient{id,secret,issuer} on JumperConfig. Because jumper's ObjectMapper sets FAIL_ON_UNKNOWN_PROPERTIES=false, the misplaced fields were silently discarded rather than rejected, leaving gatewayClient null; SpectreService.determineEnvironment then dereferences it unguarded, and does so eagerly before onErrorResume is attached, so no captured event was ever published while the control plane still reported Ready. Restore RouteListenerEntry to {issue, serviceOwner} and add the top-level GatewayClient, matching the legacy gateway in KongCeClient.appendOrUpdateListenerForRequestTransformerPlugin. Writing GatewayClient once per listener is idempotent: the client is a zone-level singleton, so every listener on a route resolves the same one. Secret is deliberately left unset. Spec-held secrets are secret-manager references resolved at point of use via secrets.Get, and RouteListenerSpec.GatewayClient carries no reference yet, so a test asserts it stays empty rather than a comment that a later contributor could "finish". Two further fixes in the same feature: - Priority was LastMileSecurity+2 = 102, colliding with LoadBalancing. Both call SetUpstream with different values and features are sorted from a map, so the effective Kong upstream flipped between reconciles. Move to 103 and use sort.SliceStable. - The jumper map is keyed by consumer alone, so a second RouteListener for the same consumer silently overwrote the first. Detect it and return an error, which aborts the Route before Kong is touched, and sort by name so the message is reproducible. The wire-format test asserts the serialized JSON, since a Go struct assertion cannot catch a field in the wrong position.
makeListenerName derives the name from consumer plus apiBasePath or eventType, omitting Provider. Two spec.listeners entries differing only by provider therefore produced one name, and the second silently overwrote the first via CreateOrUpdate, losing a declared listener with no error and leaving Status.SpectreListeners listing the same ref twice. Detect the collision and return a BlockedError, following the recordUniqueDiscriminator pattern used for exposures and subscriptions. Also apply gofmt to cmd/main.go, which places the spectrev1 import in the correct group.
Main bumped shared dependencies (k8s 0.36.2->0.36.3, logr, prometheus, golang.org/x) across the modules spectre replaces, leaving spectre's go.mod stale and breaking controller-gen.
Main now requeues after FirstSetup (#544) and writes status only when it changed (#538). The finalizer Update in the first reconcile leaves the reconciler's cached client briefly serving the stale version, so the back-to-back second reconcile failed its status write with a 409. Wrap the call in Eventually, matching how the other specs in this file already drive the reconciler.
Fix all 36 golangci-lint issues: - gci: import ordering (auto-fixed) - gocritic: hugeParam on interface-mandated signatures (nolint) - errcheck: handle or suppress diagnostic errors in test utils - gosec: annotate safe subprocess calls in test utilities - nolintlint: fix directive formatting - staticcheck: annotate kubebuilder scaffold deprecation - unparam: prefix unused interface-mandated params with _ - dupl: annotate structurally identical controller boilerplate Add spectre to .goreleaser.yaml (builds + kos) so it is included in releases alongside the other 23 modules.
xCatalitY
force-pushed
the
feat/spectre
branch
from
August 13, 2026 11:01
9c2a671 to
60f2cdd
Compare
- go.opentelemetry.io/otel/sdk v1.40.0 → v1.43.0 (CVE-2026-39883) - google.golang.org/grpc v1.79.3 → v1.82.1 (GHSA-hrxh-6v49-42gf) Aligns spectre with the versions already used by rover and gateway.
Replace the hardcoded TODO(O5) placeholder with actual zone-level gateway client resolution: - ClientId: "gateway" (zone-level singleton consumer name) - Issuer: fetched from the zone's default identity Realm status This fixes the capture path end-to-end — jumper previously received the consumer's clientId and a hardcoded prod issuer URL, causing silent NPE in SpectreService.determineEnvironment. Also moves the ApiListener nil-check before the approval step to prevent orphaned ApprovalRequest CRs for event-only Listeners that can never provision downstream resources. Adds RBAC for identity.cp.ei.telekom.de/realms (get;list;watch).
The approval CRD schema requires `spec.requester.properties` to be a non-null object. Without setting it, the ApprovalRequest creation fails with: "spec.requester.properties in body must be of type object: null" All other domains (api, agentic) call SetProperties before building. Also includes the identity/realms RBAC marker from resolveGatewayCredentials.
The janitor's CleanupAll lists owned objects using a .metadata.controller field selector. SpectreApplication and Listener were added to addKnownTypes but their field indices were never registered, causing "Index does not exist" on Rover reconcile.
xCatalitY
marked this pull request as ready for review
August 19, 2026 09:22
# Conflicts: # rover-server/internal/api/server.gen.go
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements the Spectre operator for the distributed Control Plane (DHEI-20766), enabling API traffic capture via gateway-jumper integration.
Scope:
Modules touched:
spectre/(new),gateway/,rover/,rover-ctl/,rover-server/Key design decisions:
.vNsuffix)Bug fix included: Gateway controller was setting Kong service to
/proxyinstead of/listenerwhen RouteListeners are present — Spectre capture filters never fired.Verified on real infrastructure:
Test plan
cd gateway && ENVTEST_K8S_VERSION=1.32.0 go test ./...— all passcd spectre && make build test— all passcd rover && make build test— all passcd rover-server && make build test— all passhack/spectre-e2e.sh) — 14 scenarios pass