cfapi: How to hydrate placeholder file partially

71861387 5 Reputation points
2023-05-16T13:17:26.65+00:00

Hi,

We have a requirement to partially hydrate a placeholder file based on application read request and do not want hydration of rest of the file to continue in the background. Going by the documentation on the hydration policy https://video2.skills-academy.com/en-us/windows/win32/api/cfapi/ne-cfapi-cf_hydration_policy_primary, the correct policy seems to be CF_HYDRATION_POLICY_PARTIAL but even after configuring the sync provider with the partial hydration policy, even a single byte read request from the application is leading to complete file hydration in the background which is the behaviour of CF_HYDRATION_POLICY_PROGRESSIVE policy.

Looking further, came across this document https://video2.skills-academy.com/en-us/windows/win32/cfapi/build-a-cloud-file-sync-engine#hydration-policies which provides more information on this subject and says that:
The hydration policy of a cloud file is determined at the file open time by this formula:
File hydration policy = max(app hydration policy, provider hydration policy)

and default policy is CF_HYDRATION_POLICY_PROGRESSIVE for both application and sync provider.
Also, CF_HYDRATION_POLICY_PARTIAL < CF_HYDRATION_POLICY_PROGRESSIVE, hence, even if the sync provider policy is partial, if app policy is progressive (the default one), the overall file hydration policy becomes progressive which is what we are observing.

My question is how do we set (which flag to use during file open/read calls) hydration policy at the application level so that we can set it to partial ?

Thanks,
Nishit

Windows API - Win32
Windows API - Win32
A core set of Windows application programming interfaces (APIs) for desktop and server applications. Previously known as Win32 API.
2,491 questions

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. Xiaopo Yang - MSFT 12,071 Reputation points Microsoft Vendor
    2023-06-14T02:20:25.1233333+00:00

    @71861387 ,

    If the customer isn't writing a cloud sync engine, this will not work since there are no cloud sync engines today (One Drive, iCloud, etc...) who implement partial hydration.   Here are the comments from the CFAPI devs:   I assume this question is being asked from the perspective of a user who is not an engineer for a cloud provider such as OneDrive, iCloud, etc.

    Partial hydration (streaming) a file is a feature that needs to be implemented by the cloud provider app. The Cloud Files platform has APIs in place that support streaming/partial hydration, but since there are no cloud providers that use this feature, it is not possible to partially hydrate a file today. Also, since this feature isn’t being used today, there’s a good chance that if a new provider started using it, they would discover a lot of bugs and we would need to fix them to make this work._

    If my assumption was wrong and this is being asked from the perspective of a cloud provider developer, then you just need to try switching the appropriate flags during registration and connection of the sync root. When the provider is hydrating the file, they can specify a range of the file that was hydrated using the_ TransferData params (see CF_OPERATION_PARAMETERS (cfapi.h) - Win32 apps | Microsoft Learn)