How to fix "LeaseIdMissing" or "LeaseNotPresent" Error in Synapse pipeline

Hello @Kiran

Looks like you have done some troubleshooting steps in this matter,

1. Are any other processes or applications that might be accessing the same file. If there are, can we stop those processes or applications and try deleting the file again?
2. Did you try to increase the retry count in the delete activity to allow for more attempts to delete the file or folder. This can help if the lease is released shortly after the initial attempt to delete the file or folder.
3. You can try deleting the file or folder manually using the Azure Storage Explorer or Azure Portal, is that successful?

We have been running this process for nearly 2 years and never ran into this issue before. Nothing has changed on the process or pipeline's logic.

So how are you now dealing with this situation? Did you try any manual steps (scripts) to remove the lease?

# Parameters for storage account resources
$resourceGroupName = "RGName"
$storageAccountName = "storageAccountName"
$fileShareName = "FSName"
 
# Get reference to storage account
$storageAccount = Get-AzStorageAccount `
    -ResourceGroupName $resourceGroupName `
    -Name $storageAccountName
 
# Remove share and share snapshot leases
Get-AzStorageShare -Context $storageAccount.Context | `
    Where-Object { $_.Name -eq $fileShareName } | `
    ForEach-Object {
        try {
            $leaseClient = [Azure.Storage.Files.Shares.Specialized.ShareLeaseClient]::new($_.ShareClient)
            $leaseClient.Break() | Out-Null
        } catch { }
    }

@SatishBoddu-MSFT

Are any other processes or applications that might be accessing the same file. If there are, can we stop those processes or applications and try deleting the file again?

There aren't any other processes running during the datalake load. The delete activity resides inside the ForEach loop and processes multiple tables through iterations. When I run the process for one particular table it executes fine but when I run the process for multiple tables that is when we are running into the LeaseID missing error.

Did you try to increase the retry count in the delete activity to allow for more attempts to delete the file or folder. This can help if the lease is released shortly after the initial attempt to delete the file or folder.

Per your suggestion, I did increased the retry attempts to 3 and retry intervals to 30 seconds in the delete activity. Ran the process and still ran into the same error "ErrorCode: 'LeaseIdMissing'".

You can try deleting the file or folder manually using the Azure Storage Explorer or Azure Portal, is that successful?

I did tried manually deleting the file and folder using Azure Storage Explorer and I was successful in deleting both.

So how are you now dealing with this situation? Did you try any manual steps (scripts) to remove the lease?

As mentioned, this situation surfaced couple of weeks back, tried few things such as to tweak the concurrency of the delete activity, run the process for smaller batch, run the process in debug mode to monitor if there are any overlapping issues.

This particular error occurs randomly on different files, after the issue has occurred if we navigate to that particular file/folder there wouldn't be a Lease acquired on the file/folder for us to do anything about it.

We haven't tried any manual steps to remove the lease on a file/folder because the error is random, not always on one single file/folder and lease acquisition is not visible at the surface of the application, probably something in the background that is applying the lease and not getting released when needed.