how i can upload more then 1 gb file to azure datalake?

Maulik Dave 1 Reputation point
2022-12-08T13:42:57.423+00:00

Hello every one ,

i want to upload more then one gb file from my ui application .

Let me show you flow#

UI-->microservice->will use Data Lake path and upload file

I already tried with chunk update

https://stackoverflow.com/questions/53123243/uploading-large-files-to-controller-in-chunks-using-httpclient-iformfile-always

but chunk i not working for more the 1 GB file

how my ui application will be able to upload more then 1 gb file

any help on this ?

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,410 questions
Azure Blob Storage
Azure Blob Storage
An Azure service that stores unstructured data in the cloud as blobs.
2,578 questions
ASP.NET Core
ASP.NET Core
A set of technologies in the .NET Framework for building web applications and XML web services.
4,346 questions
Azure SignalR Service
Azure SignalR Service
An Azure service that is used for adding real-time communications to web applications.
131 questions
0 comments

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. BhargavaGunnam-MSFT 28,616 Reputation points Microsoft Employee
    2022-12-09T22:33:10.567+00:00

    Hello @Maulik Dave ,
    From the below documentation page, you can use DataLakeFileClient.UploadAsync method to upload large files

    https://video2.skills-academy.com/en-us/azure/storage/blobs/data-lake-storage-directory-file-acl-dotnet#upload-a-large-file-to-a-directory 

    public async Task UploadFileBulk(DataLakeFileSystemClient fileSystemClient)  
{  
    DataLakeDirectoryClient directoryClient =  
        fileSystemClient.GetDirectoryClient("my-directory");  
  
    DataLakeFileClient fileClient = directoryClient.GetFileClient("uploaded-file.txt");  
  
    FileStream fileStream =  
        File.OpenRead("C:\\Users\\contoso\\file-to-upload.txt");  
  
    await fileClient.UploadAsync(fileStream);  
  
}

    I hope this helps. Please let us know if you have any further questions.