Deployment of the Confidential Cluster Operator

This repository contains the helper scripts to install, deploy and test the Confidential Cluster Operator development release.

The installation happens on 2 clusters, the first one where the operator is deployed and the second one is the target cluster where we want to create confidential nodes.

Installation of the operator on the external cluster

The install-operator.sh script sets up the Confidential Cluster Operator on an external cluster by first creating the catalog source in the OpenShift marketplace and then deploying the operator along with its required namespace and security context constraints.

After waiting for the operator deployment to become available, the script applies the custom resource configuration with the cluster's ingress domain.

Finally, it applies the approved image configuration and exposes the Key Broker Service and registration server as OpenShift routes, making them accessible for confidential node registration from the target cluster.

./install-operator.sh

Load image on Azure

Use the script to publish the VHD image under a user name and create the gallery

./load-image-azure.sh -u <user> -i <vhd-image> [-l <location>]

The location defaults to eastus if not specified.

Note: The image must be uploaded to the same Azure region as your target cluster. Azure Compute Gallery images are regional resources and cannot be referenced across regions. Uploading to the wrong region will cause MachineSet provisioning to fail when the nodes try to pull the image.

To find the region of your target cluster, run the following against it:

oc get machineset -n openshift-machine-api \
  -o jsonpath='{.items[0].spec.template.spec.providerSpec.value.location}'

Alternatively, you can retrieve it from the cluster infrastructure object:

oc get infrastructure cluster \
  -o jsonpath='{.status.platformStatus.azure.resourceGroupName}' | \
  xargs -I{} az group show --name {} --query location -o tsv

Pass the resulting location to the script with the -l flag, for example:

./load-image-azure.sh -u <user> -i <vhd-image> -l germanywestcentral

Create a confidential MachineSet

First, you need to apply the MachineConfiguration to point to the custom image:

oc apply -f machine-config.yaml

Afterwards, we can create a machine set for confidential nodes using the script create-machineset.sh.

This script creates a confidential computing MachineSet by:

1. Creating an ignition secret with the registration server URL
2. Converting an existing MachineSet to confidential computing configuration
3. Applying the new MachineSet

The script will:

• Create conf-ignition-secret with the registration server URL
• Disable accelerated networking (not supported on DC-series VMs)
• Replace the image with the provided confidential VM-compatible image
• Set VM size to a confidential computing compatible size (default: Standard_DC4ads_v5)
• Set userDataSecret to conf-ignition-secret
• Add securityEncryptionType: VMGuestStateOnly to the managed disk
• Configure ConfidentialVM security profile with Secure Boot and vTPM
• Set replicas to 0 for safety

Usage

First, get the registration server URL from the external cluster where the operator is installed:

oc get route register-server -n confidential-clusters -o jsonpath='{.spec.host}'

Then create the confidential MachineSet on the target cluster:

./create-machineset.sh \
  --registration-server <registration-server-url> \
  --source <existing-machineset-name> \
  --new-name <new-machineset-name> \
  --image <image-resource-id> \
  [--vm-size <vm-size>]

Example:

./create-machineset.sh \
  --registration-server register-server-confidential-clusters.apps.ci-ln-2br1qk2-1d09d.ci2.azure.devcluster.openshift.com \
  --source afrosi-test-h96rn-worker-germanywestcentral1 \
  --new-name machinset-conf-nodes \
  --image /resourceGroups/afrosi_group/providers/Microsoft.Compute/galleries/afrosi_gallery/images/rhcos-9.6.20260309-0-azure.x86_64/versions/0.1.0

N.B For the image specification, you need to provide the path but without the subscription, this will be prefixed by the machinset automatically.

The script will create and apply the MachineSet. Scale up when ready:

oc scale machineset <new-machineset-name> -n openshift-machine-api --replicas=1

Cleanup

Use the clean-operator.sh script to clean up resources from both clusters.

Clean up operator from external cluster

Switch to the external cluster where the operator is installed:

export KUBECONFIG=/path/to/external-cluster-kubeconfig
./clean-operator.sh --operator-cluster

This removes:

• Routes (kbs-service, register-server)
• Custom resources (ApprovedImage, TrustedExecutionCluster)
• Subscription and CSV
• OperatorGroup
• SecurityContextConstraints
• Namespace and CatalogSource

Clean up machinesets from target cluster

Switch to the target cluster where machinesets are deployed:

export KUBECONFIG=/path/to/target-cluster-kubeconfig
./clean-operator.sh --target-cluster

This removes:

• Confidential machinesets (scale down and delete)
• Secret (conf-ignition-secret)
• MachineConfig (99-worker-custom-image)
• Temporary files