How to set static ip for head nodes in HDInsight Cluster?

Federico Sardo 91 Reputation points
2023-05-08T20:29:37.1033333+00:00

Hi,

I am working with HDInsight cluster, the type of the cluster is Hadoop.

I am using ARM templates to create and destroy the cluster every day (8am to 6pm).

Regarding networking, I am using an azure virtual network with a subnet for hdinsight. We are using a vpn to connect to the virtual network.

Now we are adding a post script to install hue, and also we use ODBC connection to connect Power BI with hive tables.

Some time when the cluster is recreated the IP of the head nodes changes, so, this is the question, is there a way to set a static IP for the head node in the arm template?

Regards
Fede

Azure HDInsight
Azure HDInsight
An Azure managed cluster service for open-source analytics.
204 questions

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. KranthiPakala-MSFT 46,437 Reputation points Microsoft Employee
    2023-05-10T14:53:13.39+00:00

    @Federico Sardo

    Thanks for using Microsoft Q&A forum and posting your query.

    As per my conversation with internal folks, we cannot keep or specify a static IP for head nodes when creating HDInsight cluster. The IP addresses for the head nodes are assigned dynamically by Azure and can change each time the cluster is created.

    Alternate/Workaround: Since you need to connect to the head nodes using a VPN, you may explore using a DNS name instead of an IP address to connect to the head nodes. You can create a DNS name for the head nodes using Azure DNS. Create an Azure DNS zone for your domain name. Then create a record in the DNS zone for the head nodes of your HDInsight cluster and use the DNS name to connect to the head nodes instead of using an IP address. When you connect to the head nodes using the DNS name, the DNS resolver will look up the IP address of the head nodes in the DNS zone and use that IP address to establish the connection.

    By using a DNS name instead of an IP address, you can avoid the need to update the VPN configuration each time the cluster is created. Instead, you only need to update the DNS record for the head nodes with the new IP addresses when the cluster is recreated.

    Hope this info helps.

    Thank you

    If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

    1 person found this answer helpful.