Data flow keeps at state "AcquiringCompute", but in activity overview already failed.
Data flow keeps at state "AcquiringCompute", but in activity overview it already failed. The pipline therefore stays "In Progress" endlessly. The error message: "Hit unexpected exception, please retry. If the problem persists,…
Error in SOQL Query with Salesforce V2 dataset
Hi, I'm attempting to use the SOQL Query option to filter columns in the Salesforce dataset, but I'm running into the following error. Any help would be appreciated. Thanks!
Can you give me example for shared responsibility and service model related to like facebook?
I started learning azure i cannot see the big picture how it works.can i get an example like how facebook or instagram works in cloud computing based on their shared responsibility and their service model?
Input Payload is invalid, validation result - 'The stream was already consumed. It cannot be read again.'
Hello from last night i am receiving the following error on all my pipelines.
How to Implement Microsoft Authenticator in Apps like ASPNET Web
We have application projects and we have idea to implement OTP Microsoft Authenticator in our apps. On the login request there will be authentication using OTP without password. Our Apps using ASP.NET web based. And in our company had implement…
ADF-Global parameters inside Linked Service
Hi all! My company is starting to make use of DevOps pipeline to share the same code in all the productive and not productive envirorments. Before this, everything was wild. For example, the linked service for the Azure KeyVault has the url-link written…
![](https://techprofile.blob.core.windows.net/images/vsYHzBa4y02Hvuk4Yj7plg.png?8DAAA0)
Need Guidance on Deleting Folders with Azure Data Factory in ADLS
I have a pipeline designed to remove certain folders from ADLS. My folder structure is organized as…
How does Azure Data Factory's Data Copy REST API pagination rules work, is there any parallelisation and can this be disabled to only copy one page at a time?
Hello, I'm using the copy data activity in a Azure Data Factory pipeline to copy data from a REST API data source (the pipeline will soon be migrated to a Synapse one). A recent update to the source API has stumped me as using the built in pagination…
ADF Lookup Activity not returning values from Snowflake
Hi there, We have a pipeline that uses a lookup activity to run a script against Snowflake, return the first row of data and assign that returned data into a variable. The lookup code is: { "name": "Last Run", …
Column truncate issue - Copy file to DB ADF pipeline
Hi All, I have a pipeline where I am copying data from a file to a database. In the file, I have a few columns with extra spaces that I need to trim or truncate. I am not getting any option to use any transformation or function (dynamic content) on the…
How can we use insert-update function in synapse analytics notebook while storing data to blob and database
I am beginner to synapse analytics. Trying save data to blob and DB using below code. But showing "AnalysisException: /mnt/usprodannex1/usprodgold_layer/ is not a Delta table." error. Note imported required library( from delta.tables import *)…
How do I use Data Factory with private resources in an Azure Hub and Spoke topology
I'm trying to set up a Data Factory pipeline copy job between an Azure file share hosted on a storage account and azure blob storage. The file share and the blob storage are on separate subscriptions and are connected to two separate VNETs. These VNETs…
![](https://techprofile.blob.core.windows.net/images/lvNaBJBqh0eurOu4q2bQSQ.png?8DA4E8)
Mappings are not working when "Enable staging" is checked in ADF copy activity. is there a solution?
Hello, I'm using a copy data activity in azure data factory (ADF) to move data from a CSV file to SQL server. I enabled the staging in the settings of the copy activity and pointed to a blob storage, the problem is that the columns mapping is not working…
US government regulations with azure
I am looking for detailed documentation on ( Family Educational Rights and Privacy Act) FERPA and I am looking for useful information. Still, I could not get a detailed implementation of FERPA with various Azure services such as ADLS, Azure Databricks,…
How to solve issue with salesforce connector
The previous Salesforce legacy connector allowed SQL queries, which was useful for handling data issues in certain objects. The new connector only supports SOQL queries e.g.,select fields(standard) from xyz ,lacking support for custom field selection or…
Synapse Notebook running manually but not throught the pipeline in Synapse (with access policies defined in Key vaults for both my account and the Synapse Workspace))
creds = mssparkutils.credentials.getSecret('fmd-prod-db-readonly','Analytics-JSON') #Define credentials from json credentials = Credentials.from_service_account_info(json.loads(creds)) client = BetaAnalyticsDataClient(credentials=credentials) Im using…
Validation
Hi, 3+2 ; select case when count(emp) = coun(dept) then pass or fall end from emp (How to validate condition and send extract to blob storage as results.. Pls share screenshot
![](https://techprofile.blob.core.windows.net/images/5kfUQwbXfkqnY7o5wQqfdA.png?8DAA55)
How to troubleshoot Hit unexpected exception in Azure Data Factory.
Hello, Today I've been getting the following errors in my Data Factory runs: Hit unexpected exception, please retry. If the problem persists, please contact Azure Support" According to this thread:…
Getting error with ADF dataverse connector
Hi, I am using a copy activity to populate 'ownerid' column of dataverse Account entity. I got the ownerid by a lookup from dataverse 'systemuser; entity in a previous step. But I am getting the below error. Please…
![](https://techprofile.blob.core.windows.net/images/5kfUQwbXfkqnY7o5wQqfdA.png?8DAA55)
Self Hosted Integration Runtime Connection Failed with Azure Cosmos DB for PostgreSQL
In datafactory we have a step which connects with Azure CosmosDB for PostgreSQL to copy the some data to other DB using self hosted integration runtime, Its was working fine till last week. Suddenly there is intermittent connectivity issue & we are…