Getting Socket error code timed out while connecting to a SFTP server via Azure data factory
I am very new to Azure and My Use Case is that, I want to move the file from my Azure blob storage to a third party SFTP server. I got to know that Azure data factory will help me to move the files out by creating linked services from source and target…
Google BigQuery updated linked service error: "Message=No current element is available" for empty result
I started to receive failure with error "Message=No current element is available" for Copy activity with the new Google BigQuery linked service. The legacy one doesnt throw an error/failure when the Google SQL query gives an empty result. I…
ADF Pipeline Parameter Treated as Integer Instead of String When Triggered by Schedule
Hey All, Hoping you can help with a really strange issue we’re encountering with ADF pipeline parameters and scheduled triggers. We’ve got a pipeline parameter of type string that defaults to "-1" when nothing is passed through. Everything…
How do I parameterize key vaults and secrets in an ADO release pipeline for ADF
Hi, I am building a release pipeline in Azure DevOps to deploy from my Dev ADF into my Prod ADF. A lot of my linked services use a key vault with a secret to connect, however this is not exported in the ARM template parameters so I can parameterize in…
Azure data factory snowflake connector v2 connection issues
I'm working on an azure datafactory system that loads data into Snowflake using the existing / V1 Snowflake connector I saw an alert on the data factory portal warning that the connector is being retired and advising to use the Snowflake connector…
Exporting data via an OData API
I am experiencing issues while attempting to retrieve data from multiple Cornerstone OnDemand (C SOD) API views using Azure Data Factory. The process works seamlessly on Postman, but when implemented in ADF, I encounter authentication and request…
Secure Data connection from Data Factory.
Hi, We are using Data factory service in Azure and created "Selfhosted Integration Runtime" on the window virtual machine in azure for connecting on-premise data store(For ex:Db2). Using azure express route for network connectivity between…
Azure Data Factory (ADF) dataflow seems to pick the wrong endpoint when Data Preview
I have created a storage account/blob storage StorageV2 in Azure. Inside it I have created a container. Storage account automatically creates endpoints for all the provided services Then in ADF I have created a linked service (user assigned managed…
Power BI /Tableau Dataset Need to refresh
Hello Team, I have requirement, Where the Power BI/Tableau Dataset need to refresh after the ADF Pipeline ran successfully without any manual Interference. How this can be achievable Regards Rohit
To copy data from single payload to multiple tables.
I have a JSON payload containing student information stored in an Azure Blob container. The JSON includes details like StudentName, age, address, course, semester, universityName, universityAddress, and universityRating. I need to transfer this data from…
Azure Data Factory GitHub Actions getting the error message "InvalidContentLink"
I'm stuck on the following error message: Error: ERROR: {"code": "InvalidContentLink", "message": "Unable to download deployment content from '/ArmTemplate_0.jsonsig=Sanitized Azure Storage Account Shared Access…
Not able to execute Snowflake stored procedure from ADF via Script or Lookup activity
I am not able execute Snowflake Stored Procedure which accepts parameters from ADF via Script or Lookup activity. Though I know Script parameters are not supported in Script activity currently after update but also as suggestion, I have tried to use a…
Getting error on trying to call a REST API on Data Factory
Through Postman, I'm able to request a token using the following curl request - some info were hidden for security purposes. curl --location 'https://hidden/Token' \ --header 'Content-Type: application/x-www-form-urlencoded' \ --data-urlencode…
How can I run sh codes on Virtual machine with azure datafactory ?
Hello, I need to run sh codes on azure virtual machines triggered from azure datafactory. I have an application and it needs to be schedule.I will schedule it from azure data factory. Could you please help ?
How to fix web activity error "Error calling the endpoint" for Azure API management service from Azure data factory
I have an azure data factory in VNet and API management service with virtual network type "Internal". I am trying to invoke one of the Api endpoints from data factory using web activity "POST" method but getting the below mentioned…
ADF: Table type parameter in ADF copy activity source
I'm want to move data form source DB to Target DB. I'm using series of copy activity to move the data form source to target. In copy activity in the source I'm calling stored procedure and what ever SP returns that I'll dump in target DB using sink. In…
Optimizing Azure Synapse Analytics for Spark Pool and Mapping Data Flows
I am encountering errors involving Livy sessions when running multiple pipelines in Synapse Notebook and mapping data flows. The errors are as follows: For notebooks: Operation on target forecast_constrained failed: Exception: Failed to create Livy…
Json file load in adf
Hi, how to run json file on synapse workspace.. Tried with data flow but not reading files after data using flattens transformation... convert in parquet..Pls share screenshot(file attached). have selected …
Json in synapses
Hi, how to run json file on synapse workspace.. Tried with data flow but not reading files after data using flattens transformation... Trying convert in parquet..Pls share screenshot(file attached). have selected …