CSI own zone/region label values

[sergelogvinov]

Feature Request

Description

CSI requires properly set zone/region node labels, but sometimes this is not possible.
We need a feature flag that allows CSI to use its own node annotations to determine the correct zone/region values.

topology.proxmox.sinextra.dev/region
topology.proxmox.sinextra.dev/zone
topology.proxmox.sinextra.dev/node
topology.proxmox.sinextra.dev/ha-group

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request

[rdegez]

Shameless plug since this might be of interest here :
At our company, we needed a way to have k8s worker nodes topology labels automatically updated depending on the Proxmox host they are currently running on. (Since this info can change in the life of the kubernetes worker with cold or live migration)

We developed Topomatik for that matter, maybe that can be of interest here ?
In the case of Proxmox clusters, we use LLDP to dynamically pass this information from the host to the VMs.

Cheers

[jarrettprosser]

I thinks we've just come across this issue as well. We'd previously been using the VM name as the hostname (vm001) but we've now added an internal domain (vm001.internal.companyname.com). Because the node name in Kubernetes is then the hostname of the VM, vm001.internal.companyname.com, it appears that the Proxmox CSI fails to match the VM name.

proxmox-csi-plugin-controller I1028 10:03:47.456538       1 controller.go:314] "ControllerPublishVolume: called" args="{\"node_id\":\"vm001.internal.companyname.com\",\"volume_capability\":{\"AccessType\":{\"Mount\":{\"fs_type\":\"ext4\"}},\"access_mode\":{\"mode\":7}},\"volume_context\":{\"cache\":\"none\",\"storage\":\"local\",\"storage.kubernetes.io/csiProvisionerIdentity\":\"1761625888199-9286-csi.proxmox.sinextra.dev\"},\"volume_id\":\"companyname//local/vm-9999-pvc-182b70b4-9b27-4d03-9c41-0776d009c4a5\"}" 
proxmox-csi-plugin-controller I1028 10:03:47.458926       1 controller.go:351] "ControllerPublishVolume: failed to get proxmox vmrID from ProviderID" cluster="companyname" nodeID="vm001.internal.companyname.com"
csi-attacher I1028 10:03:47.464966       1 connection.go:202] GRPC response: {}
csi-attacher I1028 10:03:47.464979       1 connection.go:203] GRPC error: rpc error: code = Internal desc = vm 'vm001.internal.companyname.com' not found    

I think I can work around by forcing the hostname in the kubelet config for now, but it would be useful if I could add a label to the node to indicate the VM name or provide some other kind of override.

[sergelogvinov]

SMBIOS custom fields https://github.com/sergelogvinov/proxmox-csi-plugin/blob/main/docs/options-node.md#cloud-init-smbios-custom-fields on latest release (v0.16.0) can help too.

[chriscarpenter12]

Are the labels for topology.proxmox.sinextra.dev/region and topology.proxmox.sinextra.dev/zone supported in addition to topology.kubernetes.io/region and topology.kubernetes.io/zone? Looking at the options-node docs indicate it is but when setting these and different values on topology.kubernetes.io/region and topology.kubernetes.io/zone I see the controller failing to resolve

My pre cluster name is home

labels:
  topology.kubernetes.io/region: home
  topology.kubernetes.io/zone: home
  topology.proxmox.sinextra.dev/region: home
  topology.proxmox.sinextra.dev/zone: pve1

csi-provisioner E0322 15:53:40.596872       1 capacity.go:551] "Unhandled Error" err="CSI GetCapacity for {segment:0xc00000e060 storageClassName:proxmox-local-lvm}: rpc error: code = Internal
desc = 500 hostname lookup 'home' failed - failed to get address info for: home: Name or service not known" logger="UnhandledError"
csi-provisioner W0322 15:53:40.597990       1 capacity.go:552] Retrying capacity.workItem{segment:(*topology.Segment)(0xc00000e060), storageClassName:"proxmox-local-lvm"} after 12 failures
proxmox-csi-plugin-controller E0322 15:53:40.596624       1 controller.go:721] "GetCapacity: failed to get storage status" err="500 hostname lookup 'home' failed - failed to get address info f
or: home: Name or service not known" cluster="home" storageID="local-lvm" storageConfig={"id":"storage/pve3/local-lvm","type":"storage","Content":"images,rootdir","Disk":245947604926,"MaxDisk"
:1884655976448,"Node":"pve3","PluginType":"lvmthin","Status":"available","Storage":"local-lvm"}
proxmox-csi-plugin-controller E0322 15:53:40.596648       1 main.go:101] "GRPC error" err="rpc error: code = Internal desc = 500 hostname lookup 'home' failed - failed to get address info for:
 home: Name or service not known"

If I remove topology.kubernetes.io/region and topology.kubernetes.io/zone entirely or set the same values the controller seems to reconnect instantly.

[sergelogvinov]

@chriscarpenter12 Hello, the main error is - hostname lookup 'home' failed - failed to get address info for
probably you did not set properly CSI config file. https://github.com/sergelogvinov/proxmox-csi-plugin/blob/main/docs/config.md

the url https://cluster-api-1.exmple.com:8006/api2/json you can use hostname or IP address which points to the proxmox API server.

clusters:
  # List of Proxmox clusters
  - url: https://cluster-api-1.exmple.com:8006/api2/json
    # Skip the certificate verification, if needed
    insecure: false
    # Proxmox api token
    token_id: "kubernetes-csi@pve!csi"
    token_id_file: "/etc/proxmox/token_id"          # Optional, alternative to token_id
    token_secret: "secret"
    token_secret_file: "/etc/proxmox/token_secret"  # Optional, alternative to token_secret
    # Region name, which is cluster name
    region: Region-1