Unable to connect Fivetran with public IP to ADLS Gen2 in Vnet with Private end point

Sriram M 0 Reputation points
2024-07-01T13:39:20.48+00:00

Hi,

I'm encountering an issue while trying to connect Fivetran to ADLS Gen2 in the UK South region. Here are the details:

I have configured ADLS Gen2 as a destination in Fivetran and completed the prerequisites, including creating an SPN with appropriate access. However, I'm receiving the following error message during connection attempts: "Websicket connection blocked - A blocked connection can be caused by Adblocker browser extensions, Antivirus/Firewall software, Proxy/VPN connections or local processing agent not active."

Steps taken so far:

  • Whitelisted Fivetran's IP range (CIDR) in the storage account firewall settings.
  • Verified that there are no logs in the storage account diagnostic settings during connection attempts from Fivetran.

Since using Fivetran's private link option is cost-prohibitive for our dev/test environment, I'm seeking guidance on how to establish a successful connection using Fivetran's public IP with the storage account via a private endpoint.

Could someone please assist or suggest steps to achieve this? Any workarounds applicable to dev/test environments would be greatly appreciated.

Thank you.

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,409 questions
Azure Storage Accounts
Azure Storage Accounts
Globally unique resources that provide access to data management services and serve as the parent namespace for the services.
2,871 questions
Azure Virtual Network
Azure Virtual Network
An Azure networking service that is used to provision private networks and optionally to connect to on-premises datacenters.
2,264 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,621 questions
Azure Network Watcher
Azure Network Watcher
An Azure service that is used to monitor, diagnose, and gain insights into network performance and health.
161 questions
0 comments

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. Nehruji R 4,131 Reputation points Microsoft Vendor
    2024-07-02T11:04:53.1833333+00:00

    Hello Sriram M,

    Greetings! Welcome to Microsoft Q&A Platform.

    I understand that you would like to establish a successful connection using Fivetran's public IP with the storage account via a private endpoint and to resolve the error - "Websicket connection blocked - A blocked connection can be caused by Adblocker browser extensions, Antivirus/Firewall software, Proxy/VPN connections or local processing agent not active."

    Ensure that you’ve whitelisted Fivetran’s IP range (CIDR) in the storage account firewall settings. This allows Fivetran to access your ADLS Gen2 account. If you have a firewall enabled and your Fivetran instance is configured to run in the same region as your Azure Storage Account, consider configuring virtual network rules. Add Fivetran’s internal virtual network subnets to the list of allowed virtual networks.

    A private endpoint is a special network interface for an Azure service in your Virtual Network (VNet). When you create a private endpoint for your storage account, it provides secure connectivity between clients on your VNet and your storage. The private endpoint is assigned an IP address from the IP address range of your VNet. The connection between the private endpoint and the storage service uses a secure private link.

    Refer this following article which have detailed guidance and steps to create the Private endpoint.

    But when you enable Private endpoint, the storage doesn't allow public access at all. And hence it fails, when accessing the Storage container using Azure Portal from the virtual machine via a browser because this connection is Public and private endpoint expects the client to connect to the Private IP address.

    Refer: https://video2.skills-academy.com/en-us/azure/private-link/tutorial-private-endpoint-storage-portal?tabs=dynamic-ip#disable-public-access-to-storage-account

    Private endpoint is mainly a connection from a private IP address to a private IP address and the traffic is not public at all.

    By enabling a private endpoint, you're bringing the service into your virtual network.

    Client (subnet with private endpoint) ---> uses private IP address ------> to access the FQDN of the SQL server or any other Azure service which should resolve to the private endpoint IP address.

    Refer: https://video2.skills-academy.com/en-us/azure/private-link/private-endpoint-overview

    https://video2.skills-academy.com/en-us/azure/private-link/private-endpoint-dns#virtual-network-workloads-without-custom-dns-server

    https://github.com/MicrosoftDocs/azure-docs/issues/77513

    Hope this answer helps! Please let us know if you have any further queries. I’m happy to assist you further.

    Please "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.

    0 comments