Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,409 questions
@Hiran Amarathunga - Thanks for the question and using MS Q&A paltform.
The error message you are seeing indicates that the file /checkpoint/sources/0/rocksdb/220.zip already exists and cannot be overwritten. This can happen if the file was not properly closed or deleted before the new file was written.
To resolve this issue, you can try deleting the existing file /checkpoint/sources/0/rocksdb/220.zip and then rerun the pipeline. If you are unable to delete the file, you can try renaming it to a different name and then rerun the pipeline.
If the issue persists, you can try checking the permissions on the file and ensure that the user running the pipeline has the necessary permissions to write to the file.
Additionally, you can try checking if there are any other processes or jobs that may be accessing the file and preventing it from being overwritten.
Hope this helps. Do let us know if you any further queries.
If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.