How do i make API call's form microsoft azure databricks using a static IP address?

Rahul Kishore 0 Reputation points
2024-01-11T15:01:20.8266667+00:00

How do i make API call's form microsoft azure databricks using a static IP address?

i am doing a get request from databricks - towards external source , To be more secure and safe - we want to whitelist only this IP (static IP from databricks ) on the source system and block api calls from any other IP addresses. we have used Firewall and implemented a static IP on our databricks instance but this is costly , we wna to use something this is less expensive Thanks in advance, Rahul

Azure Firewall
Azure Firewall
An Azure network security service that is used to protect Azure Virtual Network resources.
600 questions
Azure Application Gateway
Azure Application Gateway
An Azure service that provides a platform-managed, scalable, and highly available application delivery controller as a service.
1,002 questions
Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,047 questions

2 answers

Sort by: Most helpful
Most helpful Newest Oldest
  1. Dillon Silzer 55,481 Reputation points
    2024-01-11T15:05:17.8666667+00:00

    Hi Rahul,

    I would recommend taking a look at doing this via Azure Firewall:

    Assign a single public IP for VNet-injected workspaces using Azure Firewall

    https://kb.databricks.com/cloud/azure-vnet-single-ip

    You can use an Azure Firewall to create a VNet-injected workspace in which all clusters have a single IP outbound address. The single IP address can be used as an additional security layer with other Azure services and applications that allow access based on specific IP addresses.

    If this is helpful please accept answer.

  2. ChaitanyaNaykodi-MSFT 24,231 Reputation points Microsoft Employee
    2024-01-12T17:30:20.7+00:00

    @Rahul Kishore

    Thank you for reaching out.

    Based on your response above. I think you can explore the options of using Azure NAT Gateway or Azure Load Balancer with VNET Injection enabled DataBricks Cluster.

    As documented here

    • For simple deployments, choose an egress load balancer, also called an outbound load balancer. The load balancer’s configuration is managed by Azure Databricks. Clusters have a stable public IP, but you cannot modify the configuration for custom egress needs.
    • For deployments that need some customization, choose an Azure NAT gateway. Configure the gateway on both of the workspace’s subnets to ensure that all outbound traffic to the Azure backbone and public network transits through it. Clusters have a stable egress public IP, and you can modify the configuration for custom egress needs. You can implement this solution using either an Azure template or from the Azure portal.

    Depending on your requirements I think Azure Load balancer solution can be the cheapest option but please go through the pricing documents below for additional details.

    Additional references:

    https://video2.skills-academy.com/en-us/azure/nat-gateway/troubleshoot-nat-and-azure-services#how-to-use-nat-gateway-to-connect-outbound-from-a-databricks-cluster

    https://azure.microsoft.com/en-ca/pricing/details/load-balancer/

    https://azure.microsoft.com/en-ca/pricing/details/azure-nat-gateway/

    Hope this helps! Please let me know if you have any additional questions. Thank you!

    0 comments