Do Apache Spark Pools have a Service Tag?

Andrew Chung 20 Reputation points Microsoft Employee
2024-06-28T15:07:17.05+00:00

Our company is rolling out more restrictive network security groups and I wanted to see if we could have our Apache Spark Pools' traffic be allowed via a service tag. When I tried to trace the IP address of the pools, the IPs varied widely so a static IP range would have to be very permissive.

We already have rules around the DataFactory service tag, but those IP ranges do not overlap at all with what I'm seeing the Apache Spark pools run as.

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,614 questions
0 comments

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. PRADEEPCHEEKATLA-MSFT 84,051 Reputation points Microsoft Employee
    2024-06-30T11:06:00+00:00

    @Andrew Chung - Thanks for the question and using MS Q&A platform.

    According to the official documentation: Available service tags, Azure Synapse Analytics Apache Spark pools do not have a service tag. The IP addresses of the Spark pools are not static and can vary widely, as you have observed. Therefore, it is not possible to create a network security group rule that allows traffic from a specific IP address range.

    However, you can use the Azure Synapse Analytics workspace's managed private endpoint feature to secure your Spark pools. With managed private endpoints, you can create a private endpoint for your Spark pool that is accessible only from your virtual network. This way, you can restrict access to your Spark pool to only the resources in your virtual network.

    For more details, refer to Synapse Managed private endpoints.

    Appreciate if you could share the feedback on our feedback channel. Which would be open for the user community to upvote & comment on. This allows our product teams to effectively prioritize your request against our existing feature backlog and gives insight into the potential impact of implementing the suggested feature.

    Hope this helps. Do let us know if you any further queries.

    If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

    0 comments