Sync between Azure Blob Storage and on Premise Server

ByteWay Solutions 1 Reputation point
2020-08-12T11:51:55.453+00:00

Hi,

I have issues while synchronizing files from azure file storage to an on-premise server.

The sync works perfectly from on-premise to azure blob storage but the other way around is having the problem.

Example:

When I create a folder linked to azure storage sync in server the sync works and the folder reflects on azure blob storage.

When adding files or editing a document directly in azure blob storage, the edited version or newly created directly in azure doesn't show up in an on-premise server.

Azure Files
Azure Files
An Azure service that offers file shares in the cloud.
1,213 questions
Azure Blob Storage
Azure Blob Storage
An Azure service that stores unstructured data in the cloud as blobs.
2,578 questions
{count} votes

2 answers

Sort by: Most helpful
  1. Didier3001 986 Reputation points Microsoft Employee
    2020-08-16T14:00:17.277+00:00

    Hi @ByteWay Solutions

    As of today, any change performed from the Azure Files can take up to 24 hours to be replicated on prem. This is because unlike Windows Server, Azure Files does not yet have change notifications or journaling.
    You will find more details here:
    https://video2.skills-academy.com/en-us/azure/storage/files/storage-files-faq#azure-file-sync

    I created a file directly in my Azure file share by using SMB or in the portal. How long does it take for the file to sync to the servers in the sync group?

    Changes made to the Azure file share by using the Azure portal or SMB are not immediately detected and replicated like changes to the server endpoint. Azure Files does not yet have change notifications or journaling, so there's no way to automatically initiate a sync session when files are changed. On Windows Server, Azure File Sync uses Windows USN journaling to automatically initiate a sync session when files change.
    To detect changes to the Azure file share, Azure File Sync has a scheduled job called a change detection job. A change detection job enumerates every file in the file share, and then compares it to the sync version for that file. When the change detection job determines that files have changed, Azure File Sync initiates a sync session. The change detection job is initiated every 24 hours. Because the change detection job works by enumerating every file in the Azure file share, change detection takes longer in larger namespaces than in smaller namespaces. For large namespaces, it might take longer than once every 24 hours to determine which files have changed.
    To immediately sync files that are changed in the Azure file share, the Invoke-AzStorageSyncChangeDetection PowerShell cmdlet can be used to manually initiate the detection of changes in the Azure file share. This cmdlet is intended for scenarios where some type of automated process is making changes in the Azure file share or the changes are done by an administrator (like moving files and directories into the share). For end user changes, the recommendation is to install the Azure File Sync agent in an IaaS VM and have end users access the file share through the IaaS VM. This way all changes will quickly sync to other agents without the need to use the Invoke-AzStorageSyncChangeDetection cmdlet. To learn more, see the Invoke-AzStorageSyncChangeDetection documentation.
    Note
    Changes made to an Azure file share using REST does not update the SMB last modified time and will not be seen as a change by sync.
    We are exploring adding change detection for an Azure file share similar to USN for volumes on Windows Server. Help us prioritize this feature for future development by voting for it at Azure Files UserVoice.

    --I hope this helps. Please Accept it as an answer and "Up-Vote" the answer or message(s) that helped you so that it can help others in the community looking for help on similar topics

    Regards,
    Didier3001


  2. Dylan Kennard 1 Reputation point
    2020-09-15T20:22:52.993+00:00

    I think this really helped me regarding Azure File Share to/from On-Premise Windows Server. The details are great and bookmarking for a deployment that needs this methodology for ESRI GIS Files in coming weeks/months.

    I think what it did not answer as the main post is confusing is "Can BlockBlogStorage or BlobStorage sync with On-Premise Windows Server". I have a use case there where the Browser Application needs to use the [Block]BlogStorage API functionality but the heavy data management and loading of the files relies on users not in Azure network On-Premise using Windows 10. I hope to mount the Storage more like a traditional method fo that classic UX/UI.

    So if the -MSFT staff (@Sumarigo-MSFT or @deherman-MSFT ) have any links or answers regarding that scenario it could wrap up this question. I found this post by Bing/Googling - "can blockblobstorage sync to on-prem"

    0 comments No comments