How to monitor the daily incoming Datafiles in Azure Files and save them in SQL Database

Waleed Saleem 66 Reputation points
2020-12-12T22:53:21.283+00:00

I would like to log all files that are loaded/saved into Azure storage on daily base.
I need Information such as file name and upload time to Azure storage.

Is it possible to access this information as a summary?
And how can I read them out and save them in the database, is there any provided service ?

Thanks.

Azure Files
Azure Files
An Azure service that offers file shares in the cloud.
1,285 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,681 questions
0 comments
Accepted answer
  1. Sumarigo-MSFT 46,126 Reputation points Microsoft Employee
    2020-12-25T16:41:47.38+00:00

    @Waleed Saleem Thanks for the update and letting us know the workaround steps which fixed your issue, this could also benefit other community members. If you are looking for " get the latest added file in a folder [Azure Data Factory]" You may refer to the suggestion mentioned here
    I assume you have posted the similar thread discussion in the SO forum. The file has the meta data: LAST MODIFIED, if we didn't modify or change the files, that's the time when the files were uploaded. Just for now, we can't get the actual time/date when the data was uploaded.

    Hope this helps!

    Kindly let us know if the above helps or you need further assistance on this issue.

    -------------------------------------------------------------------------------------------------

    Please don’t forget to "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.

    0 comments

1 additional answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. deherman-MSFT 37,081 Reputation points Microsoft Employee
    2020-12-14T19:36:08.423+00:00

    @Waleed Saleem
    You can create a diagnostic setting to enable resource logs for you Azure File share. You can access resource logs either as a blob in a storage account, as event data, or through Log Analytic queries. Since you are looking to do a query I would suggest setting up Log Analytics. Otherwise, you could send to a storage account. Logs appear as blobs stored to a container in the target storage account. Data is collected and stored inside a single blob as a line-delimited JSON payload. This JSON file could then be sent to your SQL server.

    Hope this helps. Let us know if you have further questions or issues.

    -------------------------------

    Please don’t forget to "Accept the answer" and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.