Copy Dataverse data into Azure SQL using Synapse Link - Initial data is not loaded

rn 0 Reputation points
2024-03-19T15:28:23.0466667+00:00

The intended setup is to link Dynamics environment to PowerApp and use Synapse Link to copy data to ADLS. From there ADF template is used to incrementally load data to Azure SQL. In short:

Dynamics -> Synapse link -> ADLS -> ADF -> ASQL

I followed word for word the setup for the Synapse Link and used the ADF template.

The data is copied to ADLS:

User's image

The pipeline runs successfully and picks up incremental data, however, the initial load never happens. Is this the expected behaviour?Any advice?

Documentation:

https://video2.skills-academy.com/en-us/power-apps/maker/data-platform/azure-synapse-link-data-lake#prerequisites

https://video2.skills-academy.com/en-us/power-apps/maker/data-platform/azure-synapse-link-pipelines?tabs=synapse-analytics

Azure SQL Database
Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,466 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,917 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,681 questions

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. Mauro Gallo 0 Reputation points
    2024-04-10T11:47:10.9666667+00:00

    Hi, the template has a bug. you can solve it:

    • On Data Flows, change the "getMaxVersion" source option query to (update 0 with -1):
    "SELECT COALESCE(MAX(versionnumber),-1) AS maxversion FROM " + $dbschema + "." + $prefix + $entity
    0 comments

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.