APIs to monitor pipleine?
Hi, Are there any APIs to monitory Pipeline, Dataflow, Activities in Azure Data Factory? The API can directly be used in .Net application.
How can I check if my parameter consists of one or more attributes?
Hi see attachment When the string of the incoming pipeline parameter contains one attribute I should use byName, otherwise byNames. That that would presume I should build two different flows, which for the rest do the same. How can…
Get Dynamics 365 OptionSet labels in Copy Activity
Is there any way to get optionset labels from a Dynamics 365 entity using a copy activity? All I can seem to get is the option number when I query an entity (using Common data services and Dynamics 365 linked services) Thanks!
Sinking to a Power BI created Data Flow using Data Factory CDM sink
Hi, I've created a Power BI Data Flow with a single entity and I would like to use Data Factory to push data to it. I am unsure what options I should be using in the Data Factory inline CDM sink. I can see I have a model.json file in the Power BI…
![](https://techprofile.blob.core.windows.net/images/MtRNO-xYr0yKecjxuKGPCA.png?8D9284)
![](https://techprofile.blob.core.windows.net/images/MtRNO-xYr0yKecjxuKGPCA.png?8D9284)
ADF "Index out of range" when reading XLSX with hidden columns
@Anonymous @Anonymous @Anonymous Hello everyone, I have a XLSX file generated by a system, and this file has 3 hidden columns on the start (columns A, B, and C). These columns aren't important and can be ignored if necessary. So, I've…
![](https://techprofile.blob.core.windows.net/images/VbZjgKr7I0qqtGE5yGwAqA.png?8D8750)
How to get the latest Json file from any Cloud source to the ADLS whenever the ADF Pipeline runs
Hi Team Whenever we run a pipeline in ADF, how we are getting the latest file from a list of files from a source and copying it into the ADLS Regards Kks
Check Version of Azure Gateway services and upate with latest version
Hi All, Where can i find my Azure gateway service information on the Azure Portal. How can i check what version it is running on and how can i find the latest version and update it? Thanks
To get all CosmosDB Mongo API collection using foreach and copy Activity
I have 10 collection in CosmosDB with Mongo API and load in BLOB storage , I can able to do for 1 collection using copy activity but cant do for all collection as I have used for each activity but not able to get all collection name in for each…
Range and Decimal Precision in Excel to CSV conversion in ADFv2
Hi All, I was exploring Excel as a Source in ADFv2 by trying to convert an excel file into CSV by using copy activity. I faced the below challenges: 1) There is a provision to provide Range of Excel file. How can we provide dynamic range. My excel files…
Pause and resume Azure Data Factory pipeline
Respected, I have a requirement where the Azure Data Factory Pipeline will insert data into a staging table, over here I want the pipeline to pause, we will be showing a UI page to user so that they can review the data that is being processed. Once…
How to check if my integration run time is upto date
Hi All, Can someone please guide me. I want to know if my integration run time is upto date and if not how to update it. Thanks
Azure data factory - column value not flowing through to sink
I have a simple data factory. Source dataset, alter row (insert or update), sink to destination table. Source and destination are Azure SQLServer I am having trouble with one column. It is defined as tinyint in both databases. When I data…
ADF Schema validation for JSON
Hi Community, I am implementing a feature which is validating a dataset schema, I know that in ADF there is a checkbox in the Mapping Dataflow section which does it automatically if columns or type of the columns do not match against the reference. …
ADF - How to identify the number of partition needed, while using SAP source?
Hi Team, My source is SAP and my table is having billions of records, i'm using CopyData activity with Partition option as "On Int" to retrieve the data faster. I'm passing "20000" as "Max partitions number", the problem…
Performance Comparison: CDS API vs. Power Automate vs. Data Factory
I'm trying to decide on which solution would be the most efficient for transferring data to a CDS database. I've found performance benchmarks for Azure Data Factory (link here) and was wondering if there was similar data available for using CDS's native…
Not able to copy entire files and folders recursively in specific folder in Azure Data Factory
Hi, I want to copy data between two adls gen2 accounts. I am unable to copy all files and subfolders in specific folder to the destination. Please check the attached screenshot which shows directory structure of my source adls account and…
Load data from a website using ADL Data pipelines
Hi All I want to load data from an excel link on a webiste. Is there a good document to show how to do it? I want to load data using ADF. Thanks
How to set up automatice trigger for azure file share
Hi All, I want to set up trigger for Azure file share. Requirement is when file is placed in azure file path it should automatically trigger the ADF pipeline to send the file to destination. Kindly provide your inputs on the above query. Thanks in…
pipeline status shows succeeded if the last activity failed.
my pipeline contains below activity if my last Stored procedure failed (marked), my pipeline status show succeeded instead failed. I added another dummy activity after that last stored procedure (after the success), then it says failed. …
Unable to connect to GCS from ADF
Hi Team When creating a linked service in ADF to connect to GCS, the connection is getting failed. we were provided with the GCS service account access id and key and URI Any other permissions need to be taken care at gcs side to establish the…