Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,903 questions
2 answers
Suggest a way to replicate data from on premise sql server 2019 to azure synaspe analytics directly
Im using an sql server in my on premise with version 2019, I need to transfer or replicate real time data to azure synapse database, so which azure service will be more effective and flexible for large data transfer
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(166.4, 81%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
1 answer
I have a Z that appears at the end of my date field
Hello, I'm trying to retrieve all the data that was modified yesterday on my ERP via an API that is accessible in Odata. However, I'm having an issue with my date field, as I get the character Z appearing at the end of the variable. I tried converting it…
asked 2024-09-26T07:43:45.01+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(41.6, 1%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAR%3C/text%3E%3C/svg%3E)
Alexis Roche
0
Reputation points
commented 2024-09-28T15:03:44.96+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(99.2, 65%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECB%3C/text%3E%3C/svg%3E)
Chandra Boorla
1,500
Reputation points Microsoft Vendor
1 answer
Error "Cannot read properties of undefined (reading '0')"
I'm using Azure Data Factory online via a browser. I'm fairly new to ADF and have yet to complete any ADF tutorial due to various problems I've encountered. I'm doing a tutorial on ADF from Udemy. I'm running a pipeline to copy data from a…
asked 2024-07-26T10:25:18.1033333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(294.40000000000003, 77%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECR%3C/text%3E%3C/svg%3E)
Chuck Roberts
105
Reputation points
commented 2024-09-28T03:37:34.6066667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(76.8, 12%, 17%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EE%3C/text%3E%3C/svg%3E)
Eugene
0
Reputation points
0 answers
Error 21352 accessing .xlsx file from blob storage in ADF
Hi there, I have a copy activity in ADF that is set to use a .xlsx file in our blob storage as its 'source'. When connecting to the .xlsx file, I can see it in adf just fine: But when trying to select a sheet within the .xlsx file, it gives me an…
asked 2024-09-25T12:09:04.9033333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(160, 57.99999999999999%, 25%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EML%3C/text%3E%3C/svg%3E)
Mike Langlois
15
Reputation points
commented 2024-09-27T21:37:53.27+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 8%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EST%3C/text%3E%3C/svg%3E)
Smaran Thoomu
15,520
Reputation points Microsoft Vendor
2 answers
Azure Data Factory, Trigger is not working for weekly and monthly schedule but works for Daily Schedule
Create a test azure pipeline, Add a trigger to it with Monthly/Weekly frequency Let the time come for trigger, the trigger won't start the Pipeline. Now shift some time in trigger Manually, the trigger will start working.
asked 2024-09-19T14:07:32.1233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(316.8, 17%, 40%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGD%3C/text%3E%3C/svg%3E)
Gupta, Dheeraj
0
Reputation points
commented 2024-09-27T21:35:38.7133333+00:00
Smaran Thoomu
15,520
Reputation points Microsoft Vendor
1 answer
Switch from Legacy BigQuery to new BigQuery Linked Service
I have a BigQuery routine that takes start date and end date as input parameter and both input parameter is declared of type String in routine. This routine is working fine without errors. I want to invoke the BigQuery routine from Azure Data Factory.…
asked 2024-09-24T10:13:48.1233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(252.8, 87%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ES%3C/text%3E%3C/svg%3E)
ShilpaGopal
100
Reputation points
commented 2024-09-27T21:23:05.3033333+00:00
Smaran Thoomu
15,520
Reputation points Microsoft Vendor
1 answer
Upgrade to new Snowflake Linked Service, buggy with Source=mscorlib
Hi, we recently upgraded our snowflake linked service to the newer version and once we released it to our production environment half of our activities started throwing Failure happened on 'Source' side. 'Type=System.Exception,Message=[Snowflake]…
asked 2024-09-26T18:34:03.21+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(281.6, 20%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMF%3C/text%3E%3C/svg%3E)
Mike Fatal
0
Reputation points
commented 2024-09-27T20:25:28.0433333+00:00
Mike Fatal
0
Reputation points
1 answer
ADF Data Flow Mapping shows 0 rows written to Dataverse sink
Running a simple set of pipelines with data flow mappings to Dataverse sinks. Data flow runs successfully, but Rows Written through interface consistently shows 0 rows. I can see the rows were written by going to the table in Dataverse.
asked 2024-09-26T16:19:01.5633333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(304, 12%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENG%3C/text%3E%3C/svg%3E)
Noah Gilbert
26
Reputation points
commented 2024-09-27T19:44:21.4733333+00:00
Noah Gilbert
26
Reputation points
1 answer
How to setup an ETL pipeline using DB that cannot be accessed via Azure Functions
I need to create a Python ETL pipeline to update a table used in a Power BI report, but I'm facing two issues. First, one of the ETL inputs is a self-hosted on-premises SQL Server database. I can connect to this SQL database within Azure Data Factory…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(307.2, 27%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ED%3C/text%3E%3C/svg%3E)
2 answers
How to reference Json fields in copy data (ADF)?
I am currently integrating an API response into a SQL Server database using Azure Data Factory. The response from the API is structured in JSON format, as shown below: The goal is to extract the id values from each batch and insert them into a SQL…
asked 2024-09-26T16:54:25.8933333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(259.20000000000005, 32%, 35%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMT%3C/text%3E%3C/svg%3E)
Max Teitelbaum
0
Reputation points
answered 2024-09-27T19:13:22.5166667+00:00
Max Teitelbaum
0
Reputation points
1 answer
What does ADF SSIS Error "Disk Full Exception Caught" mean
Hi, we are running an Azure-SSIS Type Integration Runtime in Azure Data Factory. This morning this IR suddenly got Unavailable for some reason. Fortunately we hat the Diagnostic Settings ADFSSISIntegrationRuntimeLogs enabled and I found the following…
asked 2024-09-25T10:26:26.61+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(163.2, 89%, 25%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECA%3C/text%3E%3C/svg%3E)
Cedric Ahlers
35
Reputation points
commented 2024-09-27T19:02:44.2066667+00:00
Chandra Boorla
1,500
Reputation points Microsoft Vendor
0 answers
Why are there two TenantId's returned for an ADFPipelineRun log analytics query?
I am getting duplicate results in my log analytics queries on the ADFPipelineRun table, except for the TenantId column, which has two results instead of one. The __ResourceId's are identical for both rows. I can filter on one of the TenantIds, but why…
asked 2024-09-26T15:46:02.82+00:00
louw grobler
1
Reputation point
commented 2024-09-27T19:02:08.32+00:00
Smaran Thoomu
15,520
Reputation points Microsoft Vendor
2 answers
Error Rerunning Failed Data Factory Pipeline from Failed Activity with a Secure Output Activity
Hi all, I have a pipeline in Azure Data Factory that is run each day, which is made of several execute pipeline activities. Some of these pipelines have web activities in them which use secure output. The error I am facing is that on occasion, one of…
asked 2024-09-27T03:46:18.1166667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(121.6, 85%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECW%3C/text%3E%3C/svg%3E)
Connor Welham
0
Reputation points
answered 2024-09-27T18:28:42.68+00:00
Pinaki Ghatak
4,210
Reputation points Microsoft Employee
2 answers
Data Factory Pipeline source pointing to Azure Data Lake failing
Hello: I have a Source in a Pipeline which points to a DataSet. If I test the DataSet the connection tests fine and I can preview the data. The schema is also being picked up correctly. There is an extra attribute in the first position, called Prop_0. …
asked 2024-09-24T21:26:17.78+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(57.599999999999994, 38%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EK%3C/text%3E%3C/svg%3E)
KBD
40
Reputation points
commented 2024-09-27T18:08:41.9166667+00:00
Chandra Boorla
1,500
Reputation points Microsoft Vendor
1 answer
New BigQuery connector CopyData error when there's no data
I am trying to replace the Google BigQuery connector for the newer one before the deprecation. But the new Google BigQuery connector seems to make the "Copy Data" task fail when the source is a Query and there's no data returned from it. This…
asked 2024-09-27T17:04:22.26+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(256, 91%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EFT%3C/text%3E%3C/svg%3E)
Fredy Treboux
0
Reputation points
answered 2024-09-27T17:09:24.92+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(281.6, 0%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EYG%3C/text%3E%3C/svg%3E)
youzedev@gmail.com
640
Reputation points
1 answer
ADF SSIS Integration Runtime setup issues - Custom DNS
The files needed to setup custom DNS on Azure SSIS integration runtime in AFD, from this article, is no longer available. https://video2.skills-academy.com/en-us/azure/data-factory/azure-ssis-integration-runtime-express-virtual-network-injection#dns Can…
asked 2024-09-27T07:32:45.58+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(201.6, 57.99999999999999%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHL%3C/text%3E%3C/svg%3E)
Hannelen Leirvåg
0
Reputation points
edited an answer 2024-09-27T14:56:56.74+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(246.4, 4%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESS%3C/text%3E%3C/svg%3E)
Sina Salam
10,261
Reputation points
0 answers
Migrating ADF to Salesforce v2 Dataset, how to get MAX() value of Object's LastModifiedDate
Hello, I'm migrating to the new Salesforce v2 linked service and dataset. We have a dynamic set up to load the delta from Salesforce into our SQL Server DWH every night. We loop over all Objects that we sync, and start off with executing this statement…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(224.00000000000003, 17%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EM%3C/text%3E%3C/svg%3E)
2 answers
Data Factory pipeline that only appends new and updated data
I currently have a pipeline in Data Factory that copies several tables from an Orcale database to an Azure SQL DB. The pipeline runs once per day. What is currently being done is at each run, the database and the tables under it in the Azure SQL DB, are…
asked 2021-11-24T19:49:21.727+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(233.6, 82%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3E2%3C/text%3E%3C/svg%3E)
2JK
241
Reputation points
answered 2024-09-27T12:35:24.02+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(224.00000000000003, 40%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESR%3C/text%3E%3C/svg%3E)
Swetha Rathnala
0
Reputation points
2 answers
One of the answers was accepted by the question author.
Error on sink data to MySQL - communications link failure
I've got a Pipeline with some Data Flow activities that runs in parallel. I've got the error on last night's execution and I'd like some help to understand why and how to prevent it from happenning. Job failed due to reason: at Sink 'EnviaFerias':…
asked 2024-09-19T11:24:37.3633333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(291.2, 25%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJL%3C/text%3E%3C/svg%3E)
Joffre L. A. Mota (LG lugar de gente)
25
Reputation points
accepted 2024-09-27T11:19:59.9+00:00
Joffre L. A. Mota (LG lugar de gente)
25
Reputation points
1 answer
ADF - converting list of lists into a proper JSON format
Hello, I'm pretty new to ADF and I can't wrap my head around one case. To keep it simple: I have a source (REST API response), which theoretically is JSON but it doesn't have a "proper" JSON format. It's supposed to be a simple table but it…
asked 2024-09-25T18:29:34.62+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(233.6, 22%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPS%3C/text%3E%3C/svg%3E)
Patryk Sobczak
0
Reputation points
commented 2024-09-27T11:04:01.97+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(288, 61%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
phemanth
10,330
Reputation points Microsoft Vendor