Configure network virtual appliance in Azure HDInsight

Azure Firewall FQDN tag is automatically configured to allow traffic for many of the common important FQDNs. Using another network virtual appliance requires you to configure extra features. Keep the following factors in mind as you configure your network virtual appliance:

• Service Endpoint capable services can be configured with service endpoints that results in bypassing the NVA, usually for cost or performance considerations.
• If ResourceProviderConnection is set to outbound, you can use private endpoints for the storage and SQL servers for metastores and there is no need to add them to the NVA.
• IP Address dependencies are for non-HTTP/S traffic (both TCP and UDP traffic).
• FQDN HTTP/HTTPS endpoints can be approved in your NVA device.
• Assign the route table that you create to your HDInsight subnet.

Service endpoint capable dependencies

You can optionally enable one or more of the following service endpoints, which result in bypassing the NVA. This option can be useful for large amounts of data transfers to save on cost and also for performance optimizations.

IP address dependencies

FQDN HTTP/HTTPS dependencies

You can get the list of dependent FQDNs (mostly Azure Storage and Azure Service Bus) for configuring your network virtual appliance in this repo. For the regional list, see here. These dependencies are used by HDInsight resource provider(RP) to create and monitor/manage clusters successfully. These include telemetry/diagnostic logs, provisioning metadata, cluster-related configurations, scripts, etc. This FQDN dependency list might change with releasing future HDInsight updates.

The following list gives a few FQDNs that may be needed for OS and security patching or certificate validations during the cluster create process and during the lifetime of cluster operations:

Next steps

• Use firewall to restrict outbound traffic
• Azure HDInsight virtual network architecture