Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,029 questions
1 answer
One of the answers was accepted by the question author.
How to sync/upload files from Sharepoint to Azure Blob Storage
I am currently trying to create a chatbot that has access to a Azure Blob Storage, so that I can ask questions about the files I upload. Most of the files are located in Sharepoint, so I would like the files to automatically be updated in the Blob…
asked 2024-06-27T21:41:40.9633333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(198.4, 62%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ELF%3C/text%3E%3C/svg%3E)
Luke Field
20
Reputation points
accepted 2024-07-04T02:21:18.15+00:00
Luke Field
20
Reputation points
1 answer
Error on Data Types string to nvarchar when copying data from Netsuite to Azure SQL database in ADF
I am trying to copy a table from Netsuite to an Azure SQL database using an ODBC connector from Netsuite with SuiteAnalytics Connect. Most of the tables are coming across fine, there are a couple giving this error. I realize it is something to do with…
asked 2024-01-11T21:59:43.6766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(38.4, 85%, 13%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECC%3C/text%3E%3C/svg%3E)
Chantelle Carley
0
Reputation points
commented 2024-07-03T20:40:19.1133333+00:00
susmit sarker
0
Reputation points
2 answers
ADF Dataflow Debug is not refreshing data
Hi All, I am trying to run my Dataflow pipeline which consists of an inline dataset pointing to Common Data Model and When I run my pipeline for the first time data is loaded properly to sink but it is not fetching latest data in the consecutive runs…
asked 2024-07-03T10:52:32.68+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(22.400000000000002, 93%, 12%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKJ%3C/text%3E%3C/svg%3E)
K J, Sujanya
0
Reputation points
answered 2024-07-03T19:32:38.1+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(80, 6%, 17%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESK%3C/text%3E%3C/svg%3E)
Sujanya KJ
0
Reputation points
2 answers
One of the answers was accepted by the question author.
Data Flow in Azure Data Factory has a problem with previews in debug mode
When I refresh the preview (Data preview section--> Refresh button) I can't get new data. It shows again old data! It seems it is using a cache memory and I am not able to disable it. Is there any setting to avoid this problem or is it just a bug? …
asked 2021-06-17T10:24:07.067+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(76.8, 15%, 17%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMC%3C/text%3E%3C/svg%3E)
MASSIMO CAULI
71
Reputation points
answered 2024-07-03T17:42:14.76+00:00
Sujanya KJ
0
Reputation points
1 answer
One of the answers was accepted by the question author.
I'm encountering inconsistencies with FetchXML in Dynamics 365; it fails to load all the all attributes each time I run the query. Despite setting up FetchXML to filter and retrieve all attributes, it only returns a subset. Why is this ?
Here is the query that I am using ADF copy activity: <fetch version="1.0" output-format="xml-platform" mapping="logical" distinct="false"> <entity name="@{item().tablename}"> …
asked 2024-06-26T05:05:57.9766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(3.2, 12%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENZ%3C/text%3E%3C/svg%3E)
Namrata Zalewski (She/Her)
20
Reputation points
accepted 2024-07-03T15:46:28.7066667+00:00
Namrata Zalewski (She/Her)
20
Reputation points
5 answers
Why does Data Factory Salesforce Connector not pull correct column formats?
We had a data factory pipeline connecting to salesforce using integration runtime 4.9 and everything worked perfect. We upgraded our integration runtime to 5.1 because of a sql server connector issue and now our salesforce connector does not work…
asked 2024-06-18T12:37:25.0066667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(153.6, 55.00000000000001%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBD%3C/text%3E%3C/svg%3E)
Blasko, Dan W
0
Reputation points
answered 2024-07-03T12:06:55.04+00:00
Adam VEGH
0
Reputation points
2 answers
unable to connect synapse db
Hi, Cannot connect to SQL Database. Please contact SQL server team for further support. Server: 'tcp:ag-syn-bi-group-d.sql.azuresynapse.net,1433', Database: 'bronze', User: ''. Check the linked service configuration is correct, and make sure the SQL…
asked 2024-07-02T14:07:35.56+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(204.8, 13%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVS%3C/text%3E%3C/svg%3E)
Vineet S
285
Reputation points
commented 2024-07-03T11:08:57.7566667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 8%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EST%3C/text%3E%3C/svg%3E)
Smaran Thoomu
12,105
Reputation points Microsoft Vendor
1 answer
One of the answers was accepted by the question author.
Does private link for "Portal.adf.azure.com" sub-resource is required to connect with ADF from App Service using Managed Identity in Landing Zone
We have deployed the Azure App Service and Data factory in Azure Landing Zone (Private Network). Scenario 1: We have enabled the private endpoint for data factory resource type. Using this private endpoint - private link, we copied the data from Standard…
asked 2024-06-18T06:17:50.31+00:00
Rajoli Hari Krishna
316
Reputation points
edited the question 2024-07-03T10:30:19.17+00:00
PRADEEPCHEEKATLA-MSFT
84,531
Reputation points Microsoft Employee
0 answers
Pipelines that require a scale up of the db are not scaling up
Hi, I have a couple of pipelines that have a stored procedure activity that scale up a database before the actual processing of the data starts. This has been working fine for almost a year, however in the past 2 days, the scale up times out and…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(217.60000000000002, 16%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDR%3C/text%3E%3C/svg%3E)
2 answers
How to set up ADF Python script to access an external SFTP with Firewall Exceptions (whitelist)?
I am working on an ADF pipeline. One of the steps will include a Python script that connects to an external SFTP, download some files and upload them to my Storage Account. The SFTP owner asked to share the IP that he should add to Firewall exceptions so…
asked 2024-06-24T12:08:45.5766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(99.2, 20%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3E14%3C/text%3E%3C/svg%3E)
11-4688
111
Reputation points
answered 2024-07-03T09:34:35.56+00:00
Pinaki Ghatak
2,795
Reputation points Microsoft Employee
2 answers
CopyData in ForEach
Dear all, I want to use the copydata activity in my pipeline. I have a csv file in my storage account, which has the list of the data i want to pass into the copy activity. I create a lookup activity first and then I try to use ForEach and put the copy…
asked 2024-06-27T09:48:48.74+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(134.4, 75%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EZM%3C/text%3E%3C/svg%3E)
Zhuoqiong Mo
40
Reputation points
edited a comment 2024-07-03T09:19:41.35+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(288, 61%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
phemanth
8,165
Reputation points Microsoft Vendor
1 answer
Facing issue while using ADF Copy data activity from Azure to Kusto Bugs table
Dear Team, We are experiencing an issue with data transfer from an Azure Storage Table to a Kusto Table using Azure Data Factory (ADF). Specifically, the System_AssignedTo field, which is blank in the Azure Storage Table, is being populated with…
asked 2024-06-28T09:29:30.76+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(124.80000000000001, 33%, 22%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVV%3C/text%3E%3C/svg%3E)
Vikas Verma (HCL TECHNOLOGIES CORPORATE SER)
25
Reputation points Microsoft Vendor
commented 2024-07-03T08:11:59.66+00:00
Smaran Thoomu
12,105
Reputation points Microsoft Vendor
2 answers
One of the answers was accepted by the question author.
ODBC Error [HY104] during Data Transfer to Oracle via Azure Data Factory: Character, decimal, and binary parameters cannot have a precision of zero
Hello, I am encountering an issue while transferring data to an Oracle database using Azure Data Factory copy activity. The operation fails with the following error…
asked 2024-07-01T08:57:18.64+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(89.60000000000001, 97%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ETD%3C/text%3E%3C/svg%3E)
Tommaso Di Gennaro
20
Reputation points
accepted 2024-07-03T07:52:24.07+00:00
Tommaso Di Gennaro
20
Reputation points
1 answer
One of the answers was accepted by the question author.
Data Fabric and Azure purview cost calculations
Hi friends, our team needs to provide holistic cost estimations on Azure data fabric and Azure purview. What are all the considerations that should be considered including any dependencies as such?
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(166.4, 15%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
1 answer
One of the answers was accepted by the question author.
CDM and synapse datawarehouse
Hi friends, I have some questions on CDM common data model and what use cases it best suited for ? Since CDM has attributes , entities and relationships does it fair to assume that we don't need to do data model such as star schema and snowflake schema?…
1 answer
How to add a GUEST user to my AZURE Portal with my Microsoft Azure Sponsorship Member account
Hello, I have a Microsoft Azure Sponsorship membership that expires in October 2024, I have just begun working on my concept with developers and an AI team. I need help getting GUEST users inside of my AZURE portal instead of inviting them to be a full…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,029 questions
Azure Startups
Azure Startups
Azure: A cloud computing platform and infrastructure for building, deploying and managing applications and services through a worldwide network of Microsoft-managed datacenters.Startups: Companies that are in their initial stages of business and typically developing a business model and seeking financing.
236 questions
asked 2024-06-03T14:34:02.28+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(259.20000000000005, 31%, 35%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBS%3C/text%3E%3C/svg%3E)
BOBBY SHEPPARD
0
Reputation points
edited the question 2024-07-03T04:21:33.9233333+00:00
Ryan Hill
26,866
Reputation points Microsoft Employee
0 answers
Salesforce Bulk API Connector
I am using SOQL instead of ObjectApiName. The SOQL is generated using parameter correctly. The older legacy connector works fine for the same pipeline but the new one is giving this error msg: …
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,029 questions
Azure Startups
Azure Startups
Azure: A cloud computing platform and infrastructure for building, deploying and managing applications and services through a worldwide network of Microsoft-managed datacenters.Startups: Companies that are in their initial stages of business and typically developing a business model and seeking financing.
236 questions
asked 2024-05-28T04:11:41.51+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(185.6, 57.99999999999999%, 27%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDP%3C/text%3E%3C/svg%3E)
Dev Poudel (Health)
0
Reputation points
edited the question 2024-07-03T04:20:45.23+00:00
Ryan Hill
26,866
Reputation points Microsoft Employee
6 answers
Put Blob REST API Content-Length error
I'm trying to write the XML response obtained from a REST API GET request to a file in Azure Blob Storage, using Azure Data Factory. I'd prefer to use the Copy activity to achieve this, but it doesn't support XML sinks. I've therefore been forced into…
Azure Blob Storage
Azure Blob Storage
An Azure service that stores unstructured data in the cloud as blobs.
2,579 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,029 questions
Azure Startups
Azure Startups
Azure: A cloud computing platform and infrastructure for building, deploying and managing applications and services through a worldwide network of Microsoft-managed datacenters.Startups: Companies that are in their initial stages of business and typically developing a business model and seeking financing.
236 questions
asked 2024-05-07T10:06:05.8133333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(188.79999999999998, 64%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJW%3C/text%3E%3C/svg%3E)
Jez Walters
20
Reputation points
edited the question 2024-07-03T04:18:10.28+00:00
Ryan Hill
26,866
Reputation points Microsoft Employee
0 answers
Unable to resolve Azure database for mysql name from Data Factory
I am creating a linked service from my Azure Data Factory environment, to access my Azure Database for Mysql database. I select "Azure database for Mysql", and I can see all my mysql servers. I select "From Azure subscription" as the…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,029 questions
Azure Startups
Azure Startups
Azure: A cloud computing platform and infrastructure for building, deploying and managing applications and services through a worldwide network of Microsoft-managed datacenters.Startups: Companies that are in their initial stages of business and typically developing a business model and seeking financing.
236 questions
asked 2024-05-07T07:39:15.9766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(249.60000000000002, 71%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EIS%3C/text%3E%3C/svg%3E)
Ignacio Sales
5
Reputation points
edited the question 2024-07-03T04:18:06.97+00:00
Ryan Hill
26,866
Reputation points Microsoft Employee
4 answers
Authentication type missing from linked service in Azure Data Factory when
Hi, MS released a new connector for PostgresSQL in Linked Services of Azure Data Factory v2. We could create a new linked service in the development branch that worked ok. But when we deploy the linked service to the ADF (connected to Git), the…
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,029 questions
Azure Startups
Azure Startups
Azure: A cloud computing platform and infrastructure for building, deploying and managing applications and services through a worldwide network of Microsoft-managed datacenters.Startups: Companies that are in their initial stages of business and typically developing a business model and seeking financing.
236 questions
asked 2024-04-15T12:18:05.4833333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(76.8, 66%, 17%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAR%3C/text%3E%3C/svg%3E)
Ashish Ruparel
0
Reputation points
edited the question 2024-07-03T04:15:57.73+00:00
Ryan Hill
26,866
Reputation points Microsoft Employee