Create DataFlow using .Net SDK

jigsm 236 Reputation points
2020-11-13T03:47:07.217+00:00

Respected,

I was able to create Data Factory and Pipeline with CopyActivity as mentioned in the quick tutorial.

https://video2.skills-academy.com/en-us/azure/data-factory/quickstart-create-data-factory-dot-net

I am not able to make it work when I try to add DataFlow.

Is there any tutorial on how to do so?

39553-dataflow.png

In the code as show in the image, I am creating MappingDataFlow object as assigning all the source, sink and transformation objects.

Then I am creating the ExecuteDataFlowActivity and assigning all the required parameters as show in the image.

Now, when I run this console app, I keep getting the following error.

I am not sure how to add MappingDataFlow to a collection like the way all activities are and also how to establish a link between ExecuteDataFlowActivity and
MappingDataFlow.

Any pointers will be really useful.

Error details:

Unhandled exception. Microsoft.Rest.Azure.CloudException: The document creation or update failed because of invalid reference 'df_studentimport'.
at Microsoft.Azure.Management.DataFactory.PipelinesOperations.CreateOrUpdateWithHttpMessagesAsync(String resourceGroupName, String factoryName, String pipelineName, PipelineResource pipeline, String ifMatch, Dictionary`2 customHeaders, CancellationToken cancellationToken)
at Microsoft.Azure.Management.DataFactory.PipelinesOperationsExtensions.CreateOrUpdateAsync(IPipelinesOperations operations, String resourceGroupName, String factoryName, String pipelineName, PipelineResource pipeline, String ifMatch, CancellationToken cancellationToken)
at Microsoft.Azure.Management.DataFactory.PipelinesOperationsExtensions.CreateOrUpdate(IPipelinesOperations operations, String resourceGroupName, String factoryName, String pipelineName, PipelineResource pipeline, String ifMatch)
at DataFactoryPOC.InitializeAzure.CreatePipeLine(DataFactoryManagementClient client) in C:\Users\jignesh.shah\Downloads\POCAzureDataFactory\POCAzureDataFactory\DataFactoryPOC\InitializeAzure.cs:line 361
at DataFactoryPOC.InitializeAzure..ctor() in C:\Users\jignesh.shah\Downloads\POCAzureDataFactory\POCAzureDataFactory\DataFactoryPOC\InitializeAzure.cs:line 78
at DataFactoryPOC.Program.Main(String[] args) in C:\Users\jignesh.shah\Downloads\POCAzureDataFactory\POCAzureDataFactory\DataFactoryPOC\Program.cs:line 37

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,586 questions
{count} votes

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.