Auto refresh regular Synapse dedicated SQL pool tables

pmnguye2 21 Reputation points
2020-11-09T11:18:09.713+00:00

We're using Synapse Analytics with dedicated SQL pool. We try to create some regular tables from some parquet files from Azure Data Lake Storage Gen2 to use materialzied view. Is there anyway we can have it auto refreshed when new data coming into the ADLS?

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,466 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,917 questions
0 comments
Accepted answer
  1. MartinJaffer-MSFT 26,086 Reputation points
    2020-11-10T20:49:35.5+00:00

    Hello @pmnguye2 and welcome to Microsoft Q&A.

    I think you might want to look into Event Triggers. Even triggers can be set to start a pipeline whenever a blob / gen2 file is created. You can then have the pipeline tell Synapse / SQL pool to refresh the table.

    The refresh could be done by using Copy activity to load data into SQL pool, in the pipeline triggered by the Event trigger.
    Or if you have some other method set up, the pipeline could call on a stored proc to pull the load.

0 additional answers

Sort by: Most helpful
Most helpful Newest Oldest

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.