To Deploy Azure ARM template which is greater than 4MB from one environment to another environment

Srihari B 20 Reputation points
2024-07-02T10:13:45.4633333+00:00

Hi, I was trying to move a Azure data factory from DEV to QA using ARM template which has a size of 20 MB. When i try to export the ARM template from DEV data factory and try to import it in QA data factory using Azure custom deployment, It fails to validate the template. When I further investigated, I found out that for ARM deployment, The size constraint is 4MB.

So I tried to implement GIT in my DEV data factory to deploy my factory to QA instance. I tried using Azure DevOps Release pipelines, YAML Release and GitHub Actions YAML deployment to deploy my data factory. In DevOps I got the same 4MB limitation error and in GitHub actions it was failing to validate the template. In GitHub the build was getting successful and it was going into release until deployment step which I have shown below.
1

So this was for directly deploying the ARM template which we have when we export it from Data factory.
User's image

One more proposed solution was using Linked templates to deploy it.

So I tried using Linked templates to deploy from one factory to another. In a solution we were told that we need to use 'ArmTemplate_master.json' and 'ArmTemplateParameters_master.json' to deploy it. The master will reference to all the json files which was in the folder.
User's image One roadblock was that when we try to use these two files above to deploy from DEV to QA using azure release pipeline, It asks to point to a Storage account. It asks to provide the Storage account URL and SAS token to in additional parameters for the Factory in the release pipeline. We Generated the SAS for our blob container and provided the correct blob URL but still it was failing throwing an error which says 'Sanitized SAS token'

We have a private container (with no anonymous access), so not sure if that is a problem and we need to have public container.

So what is the right was to deploy a template which is more than 4 MB either using DevOps release or GitHub by using linked templates (If that is the only option ?) and if Blob is required for the first process we only have permission to use Private container for our project.

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,028 questions
0 comments No comments
{count} votes

1 answer

Sort by: Most helpful
  1. Nandan Hegde 30,871 Reputation points MVP
    2024-07-02T12:09:39.9666667+00:00