Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,671 questions
2 answers
How to create sha256 signature for Oracle Cloud integration Rest API ?
Hi Experts, Infra: Azure Synapse -we are using pipelines / API method to get data and create a Data Warehouse. Source: We have an Oracle Cloud Infrastructure (OCI ) as source , that will output the report to Oracle Cloud Storage. OCI has a REST API…
asked 2024-07-04T03:27:38.67+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(272, 41%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECA%3C/text%3E%3C/svg%3E)
Caniut Alex
0
Reputation points
commented 2024-07-17T11:37:49.2666667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 8%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EST%3C/text%3E%3C/svg%3E)
Smaran Thoomu
12,615
Reputation points Microsoft Vendor
0 answers
When will Workflow Orchestration Manager be updated to newer Airflow versions?
The WOM has been stuck at Airflow Version 2.6.3 for a while now. The current upstream Airflow version is 2.9.2. Is there any (public) timeline on when newer versions will be available? Is there a general release schedule?
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(185.6, 19%, 27%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
2 answers
How can I add pagination as a parameter for the Azure Data Factory Rest API call to retrieve all data?
How can I add pagination(Limit and Offset ) as a parameter for the Pipelines_ListByFactory API call? https://video2.skills-academy.com/en-us/rest/api/datafactory/pipelines/list-by-factory?view=rest-datafactory-2018-06-01&tabs=HTTP I have 65 pipelines in…
asked 2024-07-16T10:03:31.7066667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(156.8, 73%, 25%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVS%3C/text%3E%3C/svg%3E)
Vegnesa, Savio
0
Reputation points
answered 2024-07-17T07:21:30.52+00:00
Vegnesa, Savio
0
Reputation points
2 answers
One of the answers was accepted by the question author.
How to fix the copy activity error while copying data from databricks delta table to datalake in csv format
There are some error tables in Databricks delta table . Those tables need to extracted as csv and load in azure data lake , inside the folder of the container. Staging has been enabled in the copy activity since it is 2 step process. Approx row count of…
asked 2024-07-11T07:54:48.48+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(300.8, 91%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESR%3C/text%3E%3C/svg%3E)
Subhadip Roy
21
Reputation points
accepted 2024-07-17T06:49:53.1066667+00:00
Subhadip Roy
21
Reputation points
1 answer
One of the answers was accepted by the question author.
Dataflow and copy activity upsert behavior
Hi, We are using the Azure Synapse to copy data from our datalakehouse (adls gen2 storage) to Dataverse tables. We're using the Dynamics 365 Linked service. To do this we are making use of a combination of data flows and copy steps, using 'upsert' as our…
asked 2024-07-16T06:47:47.7366667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(144, 21%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMS%3C/text%3E%3C/svg%3E)
Matthew Samyn
26
Reputation points
accepted 2024-07-17T06:32:42.6033333+00:00
Matthew Samyn
26
Reputation points
1 answer
One of the answers was accepted by the question author.
Lookup in adf
Hi, How to use look up table columns in copy activity and can we write sql in lookup any screenshot appreciated
2 answers
One of the answers was accepted by the question author.
Function in adf
Hi what are the functions in adf Pls help me with screenshots
asked 2024-07-17T03:51:27.6433333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(204.8, 13%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVS%3C/text%3E%3C/svg%3E)
Vineet S
425
Reputation points
answered 2024-07-17T05:15:15.9666667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(227.2, 46%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EN%3C/text%3E%3C/svg%3E)
NIKHILANETHIKUNTA
75
Reputation points Microsoft Vendor
1 answer
ADF-Global parameters inside Linked Service
Hi all! My company is starting to make use of DevOps pipeline to share the same code in all the productive and not productive envirorments. Before this, everything was wild. For example, the linked service for the Azure KeyVault has the url-link written…
asked 2023-10-01T08:53:07.1+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(240, 39%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDA%3C/text%3E%3C/svg%3E)
Dario Antolini
21
Reputation points
commented 2024-07-17T05:05:15.74+00:00
Vinodh247
13,301
Reputation points
1 answer
One of the answers was accepted by the question author.
Azure Data Factory Copy Activity convert sql string in to jobject
I have a query from Azure SQL database which returns jsonstring like below and want to insert in to Cosmos db as JSON object (not string) "{\"type\":…
asked 2024-07-12T11:22:21.11+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(172.8, 70%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVR%3C/text%3E%3C/svg%3E)
Vijay Rudrasamudram
40
Reputation points
accepted 2024-07-17T04:51:19.14+00:00
Vijay Rudrasamudram
40
Reputation points
0 answers
Copy activity failed: Failure happened on 'Source' side
Trying to get data from PGSQL to blob using copy activity give below error after query runs for ~30 min Operation on target Copy Customer failed: Failure happened on 'Source' side.…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(67.2, 65%, 16%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(288, 61%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
2 answers
Integration runtime setup, Manual setup: Download and install integration runtime Page not found
I am writing to seek assistance regarding two issues I am encountering with Azure Data Factory Integration Runtime setup. Express setup Internal Server Error: Whenever I attempt to select "Option 2: Manual setup" and proceed to "Step…
asked 2024-07-16T16:50:34.3666667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(115.19999999999999, 57.99999999999999%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGA%3C/text%3E%3C/svg%3E)
Gulam Azeemuddin
0
Reputation points
edited the question 2024-07-17T03:31:20.3266667+00:00
PRADEEPCHEEKATLA-MSFT
85,351
Reputation points Microsoft Employee
1 answer
Upsert operation in azure data factory
We want to copy data from production and send it to dev env daily. As a solution, we re thinking to create a pipeline in Azure data factory to fetch all data from source database, scramble PII information for a few columns and send it to a destination…
asked 2024-07-12T12:10:45.25+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(102.4, 27%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERJ%3C/text%3E%3C/svg%3E)
Riya Jain
0
Reputation points
commented 2024-07-16T23:17:16.4966667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(86.4, 36%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBM%3C/text%3E%3C/svg%3E)
Bhargava-MSFT
28,951
Reputation points Microsoft Employee
1 answer
How does ADF Linked Service for Azure Databricks aligns with Job Compute Policy defined inside Azure Databricks
This is a two-part Question. First Part Context: I have ADF which contains several data pipelines. Some pipeline also includes a databricks notebook as an activity. I have created Linked Service Azure Data Factory to facilitate the pipeline which…
asked 2024-07-10T10:25:16.91+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(102.4, 26%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ER%3C/text%3E%3C/svg%3E)
Ravineesh
20
Reputation points
commented 2024-07-16T23:14:23.99+00:00
Bhargava-MSFT
28,951
Reputation points Microsoft Employee
1 answer
RFC table options
Hi, i´m using azure data factory and copy step to copy data from sap. To shrink data volume i´ve tried to use the rfc table option in combination with a pipeline variable. This fails with sap error message, that it cannot parsed, when i hardcode the…
asked 2022-12-14T13:05:50.477+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(128, 1%, 22%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EML%3C/text%3E%3C/svg%3E)
Müller, André
71
Reputation points
commented 2024-07-16T14:57:28.1966667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(195.2, 49%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAB%3C/text%3E%3C/svg%3E)
Anudeep Burli
0
Reputation points
2 answers
One of the answers was accepted by the question author.
For each count dynamic batch count
I have a scenario in which i need to have provide dynamic batch count in for each loop f. Is there any way to do it. RIghtnow , I don't see any option to give it. We need to give it as hardcoded in pipeline.
asked 2022-01-22T05:48:21.423+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(163.2, 48%, 25%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAR%3C/text%3E%3C/svg%3E)
Aditya Raj
286
Reputation points
commented 2024-07-16T14:40:37.2733333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(236.8, 25%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EEN%3C/text%3E%3C/svg%3E)
Ext Nestor Penate (Admin)
0
Reputation points
2 answers
One of the answers was accepted by the question author.
how to read excel file data (stored in adls gen2 ) in Azure Data Factory pipeline
I have source excel file which is coming as showing in the format . Please help how to read excel file with that specified format . Please help . Thank You.
asked 2024-07-15T06:39:57.58+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(89.60000000000001, 88%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBS%3C/text%3E%3C/svg%3E)
Bathini, Shirish
20
Reputation points
accepted 2024-07-16T13:58:19.0633333+00:00
Bathini, Shirish
20
Reputation points
1 answer
ADF Debug pipeline : Use activity runtime
What is the difference between 'Use data flow debug session' and 'Use activity runtime' in debug in Azure Data Factory? I have a pipeline with a Lookup activity followed by a ForEach activity. A lookup file is used to store parameters, and this file is…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(96, 26%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EV%3C/text%3E%3C/svg%3E)
2 answers
adf data flow acitivity stuck in queued status when the pipelines run
when I trigger the pipelines, it got stuck in the data flow activity as queued status
asked 2024-05-07T15:40:39.77+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(300.8, 93%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EB%3C/text%3E%3C/svg%3E)
Burcu
5
Reputation points
commented 2024-07-16T11:10:20.7166667+00:00
Smaran Thoomu
12,615
Reputation points Microsoft Vendor
1 answer
One of the answers was accepted by the question author.
Data Factory's Data Flow cost structure
Hi all, I have a question regarding the cost structure for Data Factory's Data Flow. Is the charge solely per cluster/integration runtime, or are the costs doubled if two pipelines run simultaneously, I mean, both using the same cluster/integration…
asked 2024-07-10T17:56:32.1066667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(288, 11%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERN%3C/text%3E%3C/svg%3E)
Renan Nascimento
20
Reputation points
accepted 2024-07-16T09:48:50.3933333+00:00
Renan Nascimento
20
Reputation points
1 answer
How do i refrence my dataset parameter in my base url in azure copy rest activity
Good day, Is there a way i can reference my dataset parameter in my rest copy activity on the base url, am trying to create a dynamic url:
asked 2024-07-13T10:37:57.42+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(105.60000000000001, 28.999999999999996%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EOA%3C/text%3E%3C/svg%3E)
Osamwonyi Alekwe
40
Reputation points
commented 2024-07-16T09:10:42.8833333+00:00
Smaran Thoomu
12,615
Reputation points Microsoft Vendor