How to convert type of any block blob to append blob inside adls gen2? Is there any process or activity which will convert the type of my entire blob without changing the content?

Chakraborty, Nabarun (External) 0 Reputation points
2024-05-31T06:30:48.3333333+00:00

I have a json file inside a data lake gen2 storage and that json file blob type is a block blob. I want to convert that blob type to append blob through any services of azure. Need to know the process in detail.

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,409 questions
Azure Functions
Azure Functions
An Azure service that provides an event-driven serverless compute platform.
4,566 questions
Azure Blob Storage
Azure Blob Storage
An Azure service that stores unstructured data in the cloud as blobs.
2,575 questions
Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,045 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,012 questions
{count} votes

2 answers

Sort by: Most helpful
  1. Vinodh247 12,746 Reputation points
    2024-05-31T07:45:10.9433333+00:00

    Hi Chakraborty, Nabarun (External),

    Thanks for reaching out to Microsoft Q&A.

    I have a json file inside a data lake gen2 storage and that json file blob type is a block blob. I want to convert that blob type to append blob through any services of azure. Need to know the process in detail.

    Using PowerShell:

    Copy the existing block blob to a new location and specify that the destination blob should be a block blob. This process will retain all metadata from the source blob.

    Steps:

    • Connect-AzAccount command. Set your active subscription to the one containing the storage account with the append or page blobs.
    • Create the storage account context using the New-AzStorageContext command.
    • Use the Copy-AzStorageBlob command with the -DestBlobType parameter set to Block.
    • Specify the source container, source blob name, destination container, destination block blob name, and the desired block blob tier (hot/cool/archive).
    • Execute the command to copy the blob.

    Using CLI (azcopy):

    You can use the azcopy command-line tool to directly convert an existing blob to a block blob. Here’s an example command:

    azcopy copy 'https://<storage-account-name>.<blob-or-dfs>.core.windows.net/<container-name>/<append-or-page-blob-name>' 'https://<storage-account-name>.<blob-or-dfs>.core.windows.net/<container-name>/<name-of-new-block-blob>' --blob-type BlockBlob --block-blob-tier <destination-tier>
    

    https://video2.skills-academy.com/en-us/azure/storage/blobs/convert-append-and-page-blobs-to-block-blobs?tabs=azure-powershell

    For additional reading...

    https://video2.skills-academy.com/en-us/rest/api/storageservices/understanding-block-blobs--append-blobs--and-page-blobs

    https://github.com/MicrosoftDocs/azure-docs/blob/main/articles/storage/blobs/storage-blob-append.md

    Please 'Upvote'(Thumbs-up) and 'Accept' as an answer if the reply was helpful. This will benefit other community members who face the same issue.

    0 comments No comments

  2. Nehruji R 4,131 Reputation points Microsoft Vendor
    2024-06-03T12:09:39.6833333+00:00

    Hello Chakraborty, Nabarun (External),

    Greetings! Welcome to Microsoft Q&A Platform.

    As of now, converting block blobs to other blob types (such as append blobs or page blobs) within Azure is not natively supported. Refer this thread for limitations on Block blobs.

    The storage service offers three types of blobs, block blobs, append blobs, and page blobs. You specify the blob type when you create the blob. Once the blob has been created, its type cannot be changed, and it can be updated only by using operations appropriate for that blob type, i.e., writing a block or list of blocks to a block blob, appending blocks to an append blob, and writing pages to a page blob.

    refer - https://video2.skills-academy.com/en-us/rest/api/storageservices/understanding-block-blobs--append-blobs--and-page-blobs.

    Adding to above information, you can convert the append and Page blobs to Block blob by either of the above following methods via Azure PowerShell or Azure CLI or AZ copy.

    example code for conversion using Az CLI,

    azcopy copy 'https://<storage-account-name>.<blob-or-dfs>.core.windows.net/<container-name>/<append-or-page-blob-name>' 'https://<storage-account-name>.<blob-or-dfs>.core.windows.net/<container-name>/<name-of-new-block-blob>' --blob-type BlockBlob --block-blob-tier <destination-tier>
    
    

    refer - https://video2.skills-academy.com/en-us/azure/storage/blobs/convert-append-and-page-blobs-to-block-blobs?tabs=azure-powershell for detailed guidance.

    Hope this answer helps! Please let us know if you have any further queries. I’m happy to assist you further.


    Please "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.

    0 comments No comments