Enhancing Kubernetes Clusters in VMware Cloud Director with NSX ALB

In previous blogs, we discussed running Kubernetes clusters in VMware Cloud Director using the Cluster API provider (CAPVCD) or Cluster API Provider for VMware Cloud Director. This time, we will explore a new feature introduced in VCD 10.4 that enhances the management of load balancers and associate Kubernetes nodes with virtual machines in the infrastructure. This feature allows sharing an IP address among multiple virtual services, eliminating the need for arbitrary internal IP addresses and DNS translation rules.

Background

———-

In earlier versions of VCD, each virtual service could only be assigned a unique IP address. This limited the ability to expose multiple ports on the same IP address for an ingress controller, for instance. To work around this limitation, maintainers used a combination of one-arm load balancers and DNS translation rules to assign different internal IP addresses to each virtual service.

The new feature in NSX ALB allows sharing an IP address among multiple virtual services, simplifying the configuration and management of load balancers. The Cloud Provider for VCD includes a way to enable this feature, but it applies to all Kubernetes services using the type LoadBalancer.

Configuring Shared IP Addresses

——————————-

To enable shared IP addresses for Kubernetes services in VCD 10.4 or later, you need to set the `enableVirtualServiceSharedIP` flag in the configMap that contains the configuration of the CPI. The `oneArm` flag can also be used to influence the behavior of the CPI.

There are three possible scenarios with these flags:

1. Service type load balancer with multiple ports creates virtual services that share an IP from the Edge external pool.

2. Service type load balancer with multiple ports creates virtual services that share an internal IP (usually 192.168.8.x) with a NAT rule to map an IP from the Edge external pool to the internal IP.

3. Service type load balancer with multiple ports creates virtual services with different internal IPs (usually 192.168.8.x) with NAT rules to map an IP from the Edge external pool to the internal IPs.

If you have already deployed the Cloud Provider in your environment, you can edit the configMap and delete the pod to trigger its recreation so it loads the new content of the configMap. The default name for the configMap is `vcloud-ccm-configmap`. You can display its content to check what values are currently in place.

Support for Preserving Client IP

———————————–

Currently, work is underway to also support the feature to preserve client IP. This will allow the load balancer to pass through the client IP address to the backend services, enabling better visibility and traceability of client requests.

Protecting vSphere and VMware Cloud Director Workloads

—————————————————

If you are looking for options on how to protect your vSphere and VMware Cloud Director workloads, Nakivo Backup & Replication offers capabilities to back up vSphere VMs and VCD objects such as vApps, individual VMs, and vApp metadata. This ensures that remote workloads can be recovered in case of a data loss event.

Conclusion

———-

The new feature in NSX ALB allows sharing an IP address among multiple virtual services in Kubernetes clusters running on VMware Cloud Director. This simplifies the configuration and management of load balancers and eliminates the need for arbitrary internal IP addresses and DNS translation rules. Additionally, work is underway to support preserving client IP, enabling better visibility and traceability of client requests. Finally, it’s important to consider options for protecting your vSphere and VMware Cloud Director workloads, such as Nakivo Backup & Replication.