Issues to write csv files into my storage account using ADF

Felipe Regis E Silva 46 Reputation points
2020-11-20T19:30:33.3+00:00

Hello everyone,

Im trying to load csv files in my Azure Storage Account but Im getting some issues that Im not sure what they are exactly and Id like to ask some clarifications.

When I build a pipeline and a few blocks to lookup a table list from my Azure SQL DB and iterate for each table listed to load csv files this is what happens:

41535-image.png

I'm not sure why I'm getting this unkown files into my storaga instead of csv files. I was granted contributor permission for the storage account and I'm not sure if this is happening because I should have Data Factory Contributor permission.

It follows a screenshot of the pipeline built that its running fine:

41496-image.png

I added a few dynamic contents in order to read table and schema names from an Azure SQL db and also to provide name the CSV files as they are loaded into storage account into raw/exported_tables folder. See:

41497-image.png

Into my For Each block activity, I added a copy data block with the following configuration as sink:

41509-image.png

To do so, I needed to create datasets with parameters as TableName, SchemaName and FileName.

Do you have ideas of what might be wrong so I'm getting this "body" instead of a csv files into my storage container?

Thanks in advance.

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,466 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,681 questions
0 comments

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. HimanshuSinha-msft 19,476 Reputation points Microsoft Employee
    2020-11-24T01:39:39.357+00:00

    Hello @Felipe Regis E Silva ,

    Thanks for the ask and also using the forum .

    Congrats you have almost nailed the solution and when i saw the ask i thought the you have a done something wrong withb the dynamic expression , but all expression were just correct .

    I am able to repro the error and i am confident that you are not passing the Filename to the Dataset , I request you click on the 'Open" button and pass the correct file name .

    41936-1-issues-to-write-csv-files-into-my-storage-accoun.png

    Please set the value of the file name as called out below .

    41986-2-azure-data-factory.png

    I think you are passing something like @dataset() and it should be @dataset().Filename

    Let me know how ot goes .

    Thanks Himanshu
    Please do consider to click on "Accept Answer" and "Up-vote" on the post that helps you, as it can be beneficial to other community members

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.