Azure Data Factory Private Endpoint Options

M, Akhil 20 Reputation points
2024-11-05T05:02:12.79+00:00

I am trying to create private endpoints for ADF in dev and test environment. Since we are following hub and spoke topology, we are creating Private DNS zones in Hub which would be common for dev and test environments as well. There are two private dns zones available for Azure Data Factory.

1)privatelink.adf.azure.com

2)privatelink.datafactory.azure.net

Since we are using SHIR, and we want to access everything privately inside the network, it would be really helpful if we can get some clarity on the below queries.

1)Inspite of creating the 2 PE's, we are still able to access the ADF from the outside the network/internet. Is this an expected behaviour?(and blocking public access)

2)Also do we need to create both PE's in my scenario? can we create only privatelink.datafactory.azure.net and not privatelink.adf.azure.com?

3)The record being created as part of privatelink.adf.azure.com private dns zone is under the name of portal, would this be the expected behaviour? If so, please help me on how we can implement 2 ADF's with 2 portal endpoints inside the same DNS Zone in Hub.

Thanks in advance.

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,814 questions
0 comments

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. Deepanshukatara-6769 10,210 Reputation points
    2024-11-05T06:44:06.7266667+00:00

    Hello Akhil, Welcome to MS Q&A

    Answer to 1st query/question

    When you create private endpoints for Azure Data Factory and block public access, it is expected that access from outside the network would be restricted. The purpose of private endpoints is to ensure that the resources are only accessible within the specified virtual network, effectively preventing public access. However, there is a known limitation where if a customer has a private link to access one Data Factory and that Data Factory does not block public access, other customers may still access it via public means.

    Thus, if public access is blocked and private endpoints are configured correctly, external access should not be possible.

    Answer to 2nd query/question

    You do not need to create both private endpoints privatelink.adf.azure.com and privatelink.datafactory.azure.net for Azure Data Factory. The context mentions that the command communications between the self-hosted integration runtime (IR) and Data Factory can be performed securely in a private network environment through Private Link. However, it does not specify the necessity of creating both endpoints simultaneously.

    Answer to 3rd question/query

    When you create a private endpoint for the Azure Data Factory portal, it is expected that the record in the privatelink.adf.azure.com private DNS zone will be under the name "portal." This is because there is only one Data Factory portal endpoint, and creating multiple private endpoints for the portal will result in overwriting the existing DNS entry.

    To implement two Azure Data Factories (ADFs) with two portal endpoints inside the same DNS zone in the Hub, you would need to use separate DNS zones for each ADF. This way, each ADF can have its own private DNS zone without conflicts. Here are the steps:

    1. Create Separate Private DNS Zones: Create two separate private DNS zones, one for each ADF.
    2. Configure Private Endpoints: Configure the private endpoints for each ADF to use their respective private DNS zones.
    3. Link DNS Zones to Virtual Network: Link each private DNS zone to the virtual network in the Hub.

    By following these steps, you can ensure that each ADF has its own portal endpoint without overwriting the DNS entries.

    References:

    Please let us know if you have further questions

    Kindly accept if it helps

    Thanks
    Deepanshu

    0 comments

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.