Disable Public IP for VMSS on private AKS cluster over site to site VPN

Koprucu, Mert (ADV D EU TR AP&I TIA 1) 0 Reputation points
2024-08-23T07:58:22.0433333+00:00

I need help with disabling public IP for VMSS on a private AKS cluster over site-to-site VPN.

I have a site-to-site(S2S) VPN between my company and Azure Cloud, and I have also configured an AKS with a private cluster. I can connect to AKS with a private IP (10.x.x.x).

The AKS configuration is as follows:

  • 1 system node pool with two worker nodes with "Public IPs per node: Disabled"
  • 1 user node pool with three worker nodes with "Public IPs per node: Disabled"

I employ Ingress NGINX for reverse proxy and use private load balanceres to access resources like RabbitMQ server working on AKS over S2S VPN.

It creates Load Balancer named kubernetes-internal with three IPs and five rules as expected.

However, I don't expect to have one more Load Balancer named kubernetes with two "Public IP" in Frontend IP Configuration section. After I delete one public IP from Frontend IP Configuration, I can't delete the remaining one.

Because I have system node pool and user node pool, two VMSS are created. These VMSSs have public IP. If I delete "Health probe entries" and "one of Frontend IP Configuration", Public IP assignments of VMSSs are lost. However, Public IP are assigned automatically to VMSS after a while.

I don't want to have any public exposure in my AKS cluster because I use a private AKS and LoadBalancer over site-to-site VPN. Also, I find it weird that public IP assignments to VMSS are automatic even though I manually delete Health Probes and one frontend IP configuration.

How can I have a totally private AKS cluster without any public IP? Thank you.

Azure VPN Gateway
Azure VPN Gateway
An Azure service that enables the connection of on-premises networks to Azure through site-to-site virtual private networks.
1,514 questions
Azure Virtual Network
Azure Virtual Network
An Azure networking service that is used to provision private networks and optionally to connect to on-premises datacenters.
2,401 questions

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. Ganesh Patapati (Quadrant Resource LLC) 175 Reputation points Microsoft Vendor
    2024-08-23T15:10:02.53+00:00

    Hi Koprucu, Mert (ADV D EU TR AP&I TIA 1),

    Welcome to the Microsoft Q&A Platform! Thank you for asking your question here.

    NOTE: Achieving a fully private VMSS (Virtual Machine Scale Set) without any public IPs in Azure is quite challenging due to Azure's default behavior of assigning public IPs in certain scenarios.

    Refer: https://video2.skills-academy.com/en-us/azure/virtual-network/ip-services/default-outbound-access#when-is-default-outbound-access-provided

    • However, you can manage and control this by deploying the VMSS behind a NAT Gateway or Azure Firewall, which allows you to keep the instances private while managing their outbound traffic.

    Additionally, I came across a third-party article on Medium.com that describes an Architecture setup similar to what you're trying to achieve.

    Refer: https://denniszielke.medium.com/fully-private-aks-clusters-without-any-public-ips-finally-7f5688411184

    ****** While this is not official Microsoft content and we don't specifically endorse it, I thought it might be useful for your reference.

    Let me know if you have any questions or if you’d like further assistance.

    Regards,

    Ganesh Patapati

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.