How to copy files from storage account to another storage account with the same last modified date using ADF

Mahasiva, Ravi (CSW) 60 Reputation points
2024-06-20T15:58:52.0233333+00:00

Hi Team,
The request is that, when we are running ADF copy job to copy data from Landing storage account to Prod Storage account the last modified date coming as present copy time, but we want to copy the last modified date, what we have in landing storage account we want same in Prod storage account, could you please share your advice how to do this requirement using ADF.

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,408 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,991 questions
0 comments

2 answers

Sort by: Most helpful
Most helpful Newest Oldest
  1. Deleted

    This answer has been deleted due to a violation of our Code of Conduct. The answer was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.

    Comments have been turned off. Learn more

  2. Nehruji R 4,126 Reputation points Microsoft Vendor
    2024-06-21T06:21:54.13+00:00

    Hello Mahasiva, Ravi (CSW),

    Greetings! Welcome to Microsoft Q&A Platform.

    I understand that you would like to copy files from the source storage account to the destination storage account while retaining the original LastModifiedDate of the files.

    In the ADF pipeline copy activity. Initially configure the source dataset to read from the source storage account and then configure the destination dataset to write to the destination storage account. In the copy activity settings, try setting the preserveLastModified property to true. This ensures that the LastModifiedDate remains unchanged during the copy operation. refer - https://video2.skills-academy.com/en-us/rest/api/storageservices/set-blob-properties?tabs=microsoft-entra-id

    If you’re using custom code (e.g., Python), you can use the Blob Storage SDK or make direct REST API calls to read the file from the source storage account and write the same file to the destination storage account, ensuring that the LastModifiedDate property is preserved.

    refer similar thread - https://stackoverflow.com/questions/63301721/azure-datafactory-retain-lastmodified-date-when-copying-file.

    Hope this answer helps! Please let us know if you have any further queries. I’m happy to assist you further.

    Please "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.

    0 comments