From 3bf85bc0de6a2c84f73cd0f711a28b816de8a6e7 Mon Sep 17 00:00:00 2001 From: Benjamin Ritter Date: Wed, 10 Jun 2026 11:05:48 +0200 Subject: [PATCH] feat: WIP gatewayapi docs Signed-off-by: Benjamin Ritter --- .../07-migrating-to-gatewayapi.md | 188 ++++++++++++++++++ 1 file changed, 188 insertions(+) create mode 100644 docs/04-For Operators/07-migrating-to-gatewayapi.md diff --git a/docs/04-For Operators/07-migrating-to-gatewayapi.md b/docs/04-For Operators/07-migrating-to-gatewayapi.md new file mode 100644 index 00000000..c8d4f3d9 --- /dev/null +++ b/docs/04-For Operators/07-migrating-to-gatewayapi.md @@ -0,0 +1,188 @@ +--- +slug: /migrating-to-gatewayapi +title: Migrating to Gateway API +sidebar_position: 7 +--- + +# Migrating to Gateway API + +:::warning +Gateway API support in metal-stack is still in development. Changes are expected, especially around certificate management. +::: + +From version `x.y.z` metal-stack supports [Gateway API](https://gateway-api.sigs.k8s.io/) as a replacement for Kubernetes `Ingress` resources. + +This guide explains why we are moving to Gateway API, what changes for you as an operator, and how to migrate an existing metal-stack installation. + +## Motivation + +`ingress-nginx` has been deprecated and we currently depend on it. metal-stack +has control-plane components that are not served via HTTP/GRPC. Those are +exposed as TCP Services via `ingress-nginx`. As a result we are going to move +all metal-stack components to Gateway API. + +## What will change + +### Certificate provisioning and TLS termination + +Certificate provisioning and TLS termination will move into the Gateway resource. metal-stack deployed via metal-roles will expect TLS termination to happen on the Gateway. + +Exceptions for TLS termination: + +- NSQ will for now still terminate its own certificates +- The GRPC endpoint of metal-api will stay a TCPRoute and TLS termination will stay in the application. As metal-api will be superseded by metal-apiserver in the near future, we do not see the value in migrating it fully + +### Continued need for an Ingress Controller + +For some vendor dependencies you will still require an ingress controller. Examples of services still relying on Ingress are Gardener and Thanos. As ingress-nginx is EoL we recommend switching to a different Ingress Controller implementation. + +## Before you begin + +### Familiarize yourself with Gateway API + +Gateway API is more complex than Ingress and consists of multiple different resources and operator personas. The rest of the guide assumes a basic understanding of the `HTTPRoute`, `TCPRoute`, `Gateway`, `GatewayClass`. An introduction can be found [here](https://gateway-api.sigs.k8s.io/docs/introduction/) + +### Gateway API implementation requirements + +metal-stack does not require one specific Gateway API implementation. +You can choose any Gateway API implementation that provides the following features: + +- `HTTPRoute` +- `TCPRoute` + +In our [mini-lab](https://github.com/metal-stack/mini-lab/pull/299) demo environment we are running [Envoy Gateway](https://gateway.envoyproxy.io/docs/). + +### Migration Path decisions + +- Select a Gateway API implementation, e.g. [Envoy Gateway](https://gateway.envoyproxy.io/docs/) +- Certificate management: Gateway API now requires the Gateway to know + +## Migration preparation + +### Gateway deployment + +As we require `TCPRoute`s you will have to make changes to your existing Gateways, if applicable. +Still, we suggest you create a dedicated metal-stack gateway, like we do for [mini-lab](https://github.com/metal-stack/mini-lab/roles/gateway). + +We recommend provisioning a metal-stack Gateway resource as metal-stack requires multiple TCP endpoints. + +#### Example + +```yaml +apiVersion: gateway.networking.k8s.io/v1 +kind: Gateway +metadata: + name: metal-control-plane + namespace: metal-control-plane +spec: + gatewayClassName: eg + listeners: + - hostname: '*.' + name: http + port: 80 + protocol: HTTP + - hostname: '*.' + name: https + port: 443 + protocol: HTTPS + tls: + certificateRefs: + - group: "" + kind: Secret + name: metal-api-tls + mode: Terminate + - name: nsq + port: 4150 + protocol: TCP + - name: metal-api-grpc + port: 50051 + protocol: TCP + +``` + +### Certificate management + +The biggest change during the Gateway API migration is certificate management. Using Ingress Controllers the Ingress resource configured if, and using what certificates TLS connections were terminated. This changed in Gateway API, where the Gateway listener you attach the Route on decides if TLS is used and which certificate will be used for termination. + +metal-stack.io does not require any specific certificate management process. It is possible to use self-signed/private CA certificates as well as certificates public CAs. + +:::tip +Use automated certificate management. +[cert-manager](https://cert-manager.io/docs/usage/gateway/) can handle your +certificate renewal and supports Gateway API. You can use Let's Encrypt for valid +public certificates or use a private CA. + +You can issue valid public TLS certs using [Let's Encrypt](https://letsencrypt.org/2026/01/15/6day-and-ip-general-availability) against an IP address if no DNS record is available. +::: + +### TLS termination locations + +Exposed components with TLS terminated at the Gateway: + +- metal-api (excl. GRPC endpoint) +- metal-apiserver +- metal-console +- zitadel +- headscale + +Exposed components terminating TLS in the application Pod: + +- metal-api GRPC endpoint +- nsq + + + +### Migrating Components + +::: warning +The old Ingress Controller and new Gateway are going to have different IP addresses. +::: + +Deploy the `HTTPRoutes` and `TCPRoutes` for each component using metal-role. *routes and Ingress resources can be deployed at the same time. To switch over a service to the Gateway, change the DNS record of that service from pointing to the Ingress Controller to the Gateway. + +Example for zitadel, nsq and metal_apiserver. For full documentation of all services please consult the [metal-roles repository](https://github.com/metal-stack/metal-roles). + +```yaml + +zitadel_httproute_enabled: true +zitadel_httproute_parent_refs: +- name: metal-control-plane + sectionName: https + +nsq_tcproute_enabled: true +nsq_tcproute_parent_refs: +- name: metal-control-plane + sectionName: nsq + +metal_apiserver_httproute_enabled: true +metal_apiserver_httproute_parent_refs: +- name: metal-control-plane + sectionName: http +- name: metal-control-plane + sectionName: https +``` + + +#### Remaining ingress controller dependencies + +For some metal-stack dependencies you will still require an ingress controller: + +- Gardener +- Thanos + +## Verification + +Check your endpoints are still reachable on their expected host names/IP addresses. The easiest way to do that is to check with metalctl(metal-api)/metalctlv2(metal-apiserver). + +It is also recommended to check if the DNS records really resolve the new Gateway and are not accessed via the old ingress infrastructure. Use `nslookup`/`dig` to check your hostname and make sure they resolve to the same IP address as the Gateway + +## Rollback + +You can roll back to ingress any time if an unresolvable blocker is discovered during the migration. Revert the deployment variables and rerun the playbook. + +## References + +- [Gateway API](https://gateway-api.sigs.k8s.io/) +- [Envoy Gateway](https://gateway.envoyproxy.io/docs/) +- [cert-manager — Gateway API usage](https://cert-manager.io/docs/usage/gateway/) +- [mini-lab gateway role](https://github.com/metal-stack/mini-lab/roles/gateway)