Working with AZCopy 10 and Azure Storage Blob Access Tiers
As you may know, today the Azure Blob Storage offers three different access tiers that allow us to save money depending on our storage requirements: If we need our data immediately (hot), eventually (cool) or rarely (archive). More info about the features of each tier here.
Actually, we can configure an Azure Storage Account to have an access tier by default: hot or cool. Archive is not an option for an entire account though; so you just can specify if a given blob will be in archive mode.
When you mark a storage account to be cool by default, any single blob uploaded to that account will have its access tier set to cool if no other access tier is specified during the upload operation.
This means, that even when you have declared a default access tier for a storage account it can contain blobs with different access tires among them (i.e. a cool storage account can have hot and archive blobs as well).
Working with these blob features is pretty straightforward using the Azure Portal and its GUI. But what if you need to upload some millions of blobs in hot mode to a storage account that has been configured with a cool access tier?
Enter AzCopy: the legendary console tool that always has been there to help us with uploading blobs to the Azure Storage. Today, this tool is in its 10th version and in preview mode. This version represents a radical change in how we execute commands: now they are easier and cleaner to use. Its architecture has also been modified for high-performance reliable data transfers. Besides this, it now ships the ability to work with access tiers (this feature wasn’t available in previous versions because access tiers are relatively new).
So, let’s take a look to how take advantage of these new features to achieve our goal of upload blobs specifying their access tiers explicitly:
- Download the v10 (preview or further) version of AzCopy. It now consists only of one *.exe that you can save in your preferred location. You can even change its name (I did it in order to keep parallel versions: the 8.1 along the v10 preview; so now I call the latest one by typing azc10pre)
- To work with your storage account Azure requires that you either Login into the tool using Azure Active Directory or use SAS Tokens. For this example I will be using SAS tokens: There are many ways to get a SAS token. Once you have it you just need to append it at the end of the path of the blob you are working with. In a nutshell, the SAS token contains a passport to work with the storage account for a limited time and under some conditions.
- Let’s create a container for our example:
- azcopy make https://account.blob.core.windows.net/container-namesastoken
- Observe how we have appended the sas token.
- azcopy make https://account.blob.core.windows.net/container-namesastoken
- Now let’s use the CP command to copy two files from our computer to the cloud:
- azcopy cp "C:\local\path\myfile" "https://account.blob.core.windows.net/container/myfile sastoken”
- If we operate in this way the resulting uploaded blob, will be in cool tier as it is the account´s default and we didn’t specified anything else. But, if we need to override that default, we should use:
- azcopy cp "C:\local\path\myfile" "https://account.blob.core.windows.net/container/myfile sastoken ” --block-blob-tier="hot"
- azcopy cp "C:\local\path\myfile" "https://account.blob.core.windows.net/container/myfile sastoken”
So this is the way you can have mixed access tiers inside a given storage account using the AzTool that works for Windows, Linux and MacOS