Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,133 questions
1 answer
ADF Copy Issue - Long File Path names
Hello, I am using copy activity to copy the files from on premise to Azure Storage Account . The job is failing as there is a long file path name at the source. can someone help me how can i fix this issue I am getting the following error …
asked 2020-07-27T15:06:24.047+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(83.2, 75%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKN%3C/text%3E%3C/svg%3E)
Krishna Nagesh Kukkadapu
21
Reputation points
commented 2020-07-31T20:47:48.053+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(134.4, 9%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKM%3C/text%3E%3C/svg%3E)
KranthiPakala-MSFT
46,442
Reputation points Microsoft Employee
1 answer
Azure Data Factory Linked Service to On-Premises SQL Server with named instance
Hello, I'm trying to connect from Azure Data Factory to an on-premises SQL Server using the SQL Server linked service (e.g. servername\instance1). I already have an integration runtime installed, configured, and working (tested with other servers). …
asked 2020-07-23T15:14:12.393+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(38.4, 56.00000000000001%, 13%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EN%3C/text%3E%3C/svg%3E)
Nestor
46
Reputation points
commented 2020-07-31T20:45:07.17+00:00
KranthiPakala-MSFT
46,442
Reputation points Microsoft Employee
1 answer
One of the answers was accepted by the question author.
Pipeline fails at data flow
I have created a 'copy data' and 'data flow' and have added them to my pipeline. When debugging the pipeline, copy data runs successfully, however, it fails at data flow with error: {"message":"at Sink 'OutputStreamName':…
asked 2020-07-30T08:51:55.913+00:00
Rose Kipgen
26
Reputation points Microsoft Employee
commented 2020-07-31T12:51:27.627+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(108.80000000000001, 9%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHM%3C/text%3E%3C/svg%3E)
HarithaMaddi-MSFT
10,136
Reputation points
2 answers
One of the answers was accepted by the question author.
Azure data factory copy activity cosmos db error
Hello I'm getting the following mistake when I try to make a mapping of an array on copy data activity from csv file to cosmos db. The structure on cosmos db for the item should be as follows: "InsuredItems": [ { "Accomodation":…
asked 2020-07-22T17:19:38.367+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(96, 71%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECG%3C/text%3E%3C/svg%3E)
CORONADO GRANADOS Diana Milena
126
Reputation points
accepted 2020-07-31T06:28:17.603+00:00
CORONADO GRANADOS Diana Milena
126
Reputation points
1 answer
REST API JSON DATE FORMAT
I am trying to copy data from rest api source using the azure copy activity. I have used Rest Api as source and AZ SQL DB as target. But the json response I am receiving is having the date as below format: {Createddate: /date(345667999)/} But when I…
asked 2020-07-16T13:59:05.157+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(281.6, 0%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMD%3C/text%3E%3C/svg%3E)
Mark de Jonge
1
Reputation point
commented 2020-07-30T18:49:37.733+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(192, 26%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHM%3C/text%3E%3C/svg%3E)
HimanshuSinha-msft
19,381
Reputation points Microsoft Employee
1 answer
One of the answers was accepted by the question author.
How to insert data in Azure Synapse Analytics through Azure Data factory without PolyBase
I have tried using Move and Transfar activity in data factory but I'm getting 2 error at publishing: ' Data flow activity 'dataflow1' requires a stagingLinkedService Data flow activity 'dataflow1' requires a stagingStoragePath ' I don't want to…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(185.6, 95%, 27%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMD%3C/text%3E%3C/svg%3E)
1 answer
dual factor authentication in Data Factory
Hi We have a plan to setup Azure data factory to connect sftp , for this we require dual factor authentication that means both sftp user id and ssh public key exchange method . so is this possible in current Azure Data factory V2 ? I observed…
asked 2020-07-23T05:51:50.72+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(233.6, 63%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDS%3C/text%3E%3C/svg%3E)
Devara, Santosh (Cloud Architect)
1
Reputation point
commented 2020-07-30T06:15:24.23+00:00
KranthiPakala-MSFT
46,442
Reputation points Microsoft Employee
1 answer
One of the answers was accepted by the question author.
How to copydata from RestApi using datafactory and save it in Datalake?
Hi Everyone, I'm trying to fetch data from REST API and save the json string it into DataLake and I'm getting an error. I've followed the steps mentioned here https://video2.skills-academy.com/en-us/azure/data-factory/connector-rest &…
asked 2020-07-27T11:18:38.973+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(32, 19%, 13%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ES%3C/text%3E%3C/svg%3E)
stramzik
161
Reputation points
commented 2020-07-29T17:20:33.883+00:00
KranthiPakala-MSFT
46,442
Reputation points Microsoft Employee
1 answer
Azure Data Factory- Copy activity retry
Retry of copy activity in ADF pipelines has been configured. However, I would like to capture this dynamically. What variables can I use to check if retry was done or not and capture retry run ids? Thanks in advance.
asked 2020-07-22T18:57:59.947+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(22.400000000000002, 74%, 12%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKM%3C/text%3E%3C/svg%3E)
Kay Maddy
6
Reputation points
commented 2020-07-29T05:05:56.07+00:00
HarithaMaddi-MSFT
10,136
Reputation points
2 answers
Future Roadmap for Azure Blob, Azure Data factory and Azure SQL DB
Where I can get future roadmap of below products Azure Blob Azure data factory Azure SQL DB Power BI
asked 2020-07-10T10:09:44.33+00:00
Sanjeev KUMAR
27
Reputation points
commented 2020-07-28T08:40:21.35+00:00
Sumarigo-MSFT
45,406
Reputation points Microsoft Employee
0 answers
ADF Pipeline error
I have had a pipeline running for the past month on a daily basis. Today the pipeline failed saying that the notebook does not exist. I am able to access the said notebook and it is in the proper location that the Pipeline has defined it and I can…
asked 2020-07-20T22:28:50.037+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(147.20000000000002, 30%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EB%3C/text%3E%3C/svg%3E)
Brittany
1
Reputation point
commented 2020-07-27T08:00:11.84+00:00
HarithaMaddi-MSFT
10,136
Reputation points
1 answer
One of the answers was accepted by the question author.
Cross-tenant Azure Data Factory
Hi, I'm trying to grant permission from Integration runtimes from one ADF to another ADF, but it's failing with the following error message: Error occurred when grant permission to XXXXXXXXXXXXXXXXXXXXXXX. Error:…
asked 2020-07-24T07:53:07.633+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(112.00000000000001, 83%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERA%3C/text%3E%3C/svg%3E)
Rafael Arnosti
21
Reputation points
accepted 2020-07-27T07:48:06.597+00:00
Rafael Arnosti
21
Reputation points
1 answer
How to extract data from using linkedin learning api?
How do I extract the individual information of how may courses finished? how long he spent time on course?.. By using linked in rest api how to pass parameters to get the above information? I have authentication key to access those endpoints.
asked 2020-07-21T12:39:39.463+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(16, 54%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ETG%3C/text%3E%3C/svg%3E)
THIMMAIAH GARI,PRASHANTH,,
201
Reputation points
commented 2020-07-25T00:28:38.32+00:00
KranthiPakala-MSFT
46,442
Reputation points Microsoft Employee
1 answer
azure data factory excel files (new feature)
Hello I'm creating a pipeline that uses an Input file an excel, when I create the dataset looks fine but when I've try to execute some actions to the number colums they put a strange format. The columns on the originalexcel looks like …
asked 2020-07-16T14:24:53.59+00:00
CORONADO GRANADOS Diana Milena
126
Reputation points
commented 2020-07-24T20:37:15.137+00:00
HimanshuSinha-msft
19,381
Reputation points Microsoft Employee
1 answer
Source settings missing in azure data flow
I cannot configure source in the azure data flow as the source settings tab doesnot show up anything. I have a csv uploaded in the blob storage container
asked 2020-07-09T02:45:22.143+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(92.8, 23%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENN%3C/text%3E%3C/svg%3E)
Namitha Narayanan
1
Reputation point
commented 2020-07-24T06:53:23.373+00:00
PRADEEPCHEEKATLA-MSFT
85,351
Reputation points Microsoft Employee
2 answers
Azure Data Factory - Delta Format Vaccum error
Getting this error while executing ADF using delta format ,Dont know where to set this property ? Operation on target dataflow1 failed: {"StatusCode":"DFExecutorUserError","Message":"at Sink 'sink1':…
asked 2020-07-20T05:52:08.297+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(252.8, 49%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EB%3C/text%3E%3C/svg%3E)
b_d21
1
Reputation point
commented 2020-07-23T06:49:54.663+00:00
KranthiPakala-MSFT
46,442
Reputation points Microsoft Employee
1 answer
One of the answers was accepted by the question author.
Pros and Cons between Logic Apps and Azure Data Factory specifically in relation within a Private Lake. Any Preference???
Pros and Cons between Logic Apps and Azure Data Factory specifically in relation within a Private Lake. Any Preference???
asked 2020-07-10T15:08:15.58+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(163.2, 93%, 25%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAN%3C/text%3E%3C/svg%3E)
Abhijit Nath
21
Reputation points
accepted 2020-07-23T05:04:28.76+00:00
Abhijit Nath
21
Reputation points
1 answer
edit an azure data factory pipeline
can not edit an existing pipeline, edit option is not available when right click the pipeline on Resource Explorer blade.
asked 2020-07-17T02:59:38.667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(204.8, 94%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPH%3C/text%3E%3C/svg%3E)
Pham Hong Phuc
1
Reputation point
commented 2020-07-22T04:52:35.4+00:00
HarithaMaddi-MSFT
10,136
Reputation points
0 answers
Azure Data Factory - Turn off customErrors in web.config
We conducted a penetration test and encountered the attached error. Please advise how to turn off the customErrors mode. Thanks.
asked 2020-06-12T08:31:52.663+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(230.39999999999998, 53%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAJ%3C/text%3E%3C/svg%3E)
Ang, Jianyun
1
Reputation point
commented 2020-07-22T01:54:51.297+00:00
Ang, Jianyun
1
Reputation point
1 answer
Data Factory and bad performances with REST API GET requests (SFCC API)
HI I'm facing very long response time to acquire Salesforce commerce cloud data (via REST API) The principle is to get a list of orders (activity lookup) and iterate on the ids by invoking an APi URL (for each activity) And in the end to make…
asked 2020-07-10T09:01:13.137+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(211.20000000000002, 90%, 30%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBM%3C/text%3E%3C/svg%3E)
Bamak Mrbi
31
Reputation points
commented 2020-07-20T23:25:17.577+00:00
HimanshuSinha-msft
19,381
Reputation points Microsoft Employee