Firewall Configuration for Custom Model Serving in Azure Databricks

Hello Pejman Memar,

Welcome to the Microsoft Q&A and thank you for posting your questions here.

I understand that you are having trouble with the firewall settings of your Azure Storage account based on your explanation.

This prescribed solution was based on the scenario given and your questions, while focusing on the problem statement. Therefore, endeavor to the followings:

Configure Network Access: Restrict access to your storage account by specifying allowed IP addresses, IP ranges, subnets, and Azure resource instances. This ensures that only requests from these specified networks can access the storage account.

Use Private Endpoints: Set up private endpoints for your storage account. This assigns a private IP address from your virtual network to the storage account, securing traffic within your network and preventing exposure to the internet.

Ensure Proper Authorization: Any application accessing the storage account under network rules must have proper authorization. Use Microsoft Entra credentials, account access keys, or SAS tokens to authorize requests to blobs, tables, file shares, and queues.

Disable Anonymous Access: By default, anonymous access to containers is disabled, requiring authorization for every request. You can enable anonymous access only if permitted by the storage account settings, ensuring better control over who can access the data.

For all the above use the following links for more details:

Enable firewall support for your workspace storage account - Azure Databricks

Configure Azure Storage firewalls and virtual networks

Configure anonymous read access for containers and blobs.

Create a custom policy for storage account to deny public access and allow selected networks bypassing the Azure Services

Possible to Restrict Access to Azure Container Instance with IP restrictions.

Public access is not permitted on this storage account.

Accept Answer

I hope this is helpful! Do not hesitate to let me know if you have any other questions.

** Please don't forget to close up the thread here by upvoting and accept it as an answer if it is helpful ** so that others in the community facing similar issues can easily find the solution.

Best Regards,

Sina Salam

Hello Pejman Memar,

Greetings! Welcome to Microsoft Q&A Platform.

To configure your Azure Storage account firewall for custom model serving without setting it to public access, you can set the storage account to allow traffic only from specific virtual networks, select “Enabled from selected virtual networks and IP addresses” in Network settings.

By enabling access only from specific virtual networks, you can restrict traffic to your storage account while still allowing custom model serving. Make sure to configure the appropriate virtual networks and IP addresses to ensure that your custom LLM model endpoint works as expected. Remember that authorization is still required for requests to access the storage account, even when network rules are in effect. You can use Microsoft Entra credentials, account access keys, or shared access signatures (SAS) tokens for proper authorization.

When you create a new Azure Databricks workspace, an Azure storage account is created in a managed resource group, known as the workspace storage account. The workspace storage account includes workspace system data (job output, system settings, and logs), DBFS root, and in some cases a Unity Catalog workspace catalog. This article describes how to limit access to your workspace storage account from only authorized resources and networks using an ARM template.

refer - https://video2.skills-academy.com/en-us/azure/databricks/security/network/storage/firewall-support#--what-is-firewall-support-for-your-workspace-storage-account

If you encounter any issues, check the build logs for additional details.

Hope this answer helps! Please let us know if you have any further queries. I’m happy to assist you further.

Please "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.