How to set ACL to folder and all its children recursively?

Question

I'm using the following code to set a ACL to a folder and its children in Azure Storage Account StorageV2. We have more than 2 million files and this takes too long. I could't find any recursive parameter or a best aproach. Is this the best option to give ACLs recursively to a folder? How can I grant access to a specific folder?

$MaxReturn = 500
$Total = 0
$Token = $Null
do
 {
     $items = Get-AzDataLakeGen2ChildItem -FileSystem $container -Path $filePath -Context $ctx -Recurse -MaxCount $MaxReturn  -ContinuationToken $Token
     $Total += $items.Count
     if($items.Length -le 0) { Break;}
     $Token = $items[$items.Count -1].ContinuationToken;

       foreach ($path in $items) {
        if( $path.IsDirectory ){
            Update-AzDataLakeGen2Item -Context $ctx -FileSystem $container -Path $path.Path -Acl $acl_dir
        }else{
            Update-AzDataLakeGen2Item -Context $ctx -FileSystem $container -Path $path.Path -Acl $acl
        }
     }

    Echo "Current count: $Total"
 }
While ($Token -ne $Null)
Echo "Total $Total items in Filesystem $container"

Answer

Hello @DanielArgelles-5927 and thank you for your question.
If I understand you correctly, you are currently using the provided code to set the ACL's but is taking too long.

Given that the bottleneck is on your process sending only one http request at a time, and the azure side can handle multiple requests simultaneously, one solution would be multi-threading on your end so you can send multiple requests in parallel.

Refactor, so you have one process gathering the list of files, (currently handled by your do-loop). Multiple other processes then consume filepaths from the list and return status codes.

Answer

"How can I grant access to a specific folder?"

To grant access to a directory:
Grant execute permission on each directory on the path from root to the target directory. This includes the container and the target directory.

To grant ability to see the contents of a target directory:
Grant execute and read permissions for the target directory.

To grant ability to create items in the target directory:
Grant execute and write permissions for the target directory.
If write permissions are not granted on the files, you will be able to create new files, but not edit existing ones.

To grant the ability to read the files in the target directory:
Grant execute permissions on the target directory. Also grant read permissions on all the desired items in the directory.

Answer

@DanielArgelles-5927

I have some great news!

There is a recommended recursive script for you to use. See https://github.com/jamesbak/recursive_acl

I recommend you use this until a recursive option, like you were looking for, becomes available.

Share via

How to set ACL to folder and all its children recursively?

3 answers