Azure VM Resizing and SKU Change in ASM and ARM

  • Artigo

One of the major benefit of the Cloud is the possibility to dynamically change the type and size of your VMs (scale-up/down) and have more/less power to support your workload and at the same time reducing the cost for the resources you will use. Azure provides several VM SKUs and many different sizes, each one with specific hardware capabilities in terms of CPU, RAM, networking, ephemeral and persistent storage. Depending on several factors, there are rules that govern the possible transitions between a certain SKU and the others, and this is what I’m going to explain in this blog post. Before going into details, let me introduce some basic concepts that will play a crucial role in my explanation:

  • Azure uses different Cluster types in its datacenter, each one supporting specific hardware capabilities such as CPU types, NIC types, memory, Premium or non-Premium capable hosts.
  • Azure provides two different types of management APIs:
    • Azure Service Management (ASM): this is the first API layer introduced since the beginning and it is sometimes qualified as “legacy” or “classic”:

Service Management REST API Reference

https://msdn.microsoft.com/en-us/library/azure/ee460799.aspx

  • Azure Resource Manager (ARM): this is the new API model we recently introduced as the new management layer, you should use it going forward:

Azure Resource Manager REST API Reference

https://msdn.microsoft.com/en-us/library/azure/dn790568.aspx

  • If you want to use Azure Premium storage, you need to use a VM SKU that is able to support it: actually only DS-SERIES and GS-SERIESS can do that;

In the remaining part of this blog post, I will introduce you to the differences between ASM and ARM and how each API will affect changing VM SKUs and sizes. Then, I will finally consider the impact of the storage type (Premium vs. Standard).

WARNING: Changing the size and/or SKU of a VM will cause loss of content on the temporary drive (“D:” on Windows, “/sdb” on Linux). This is an expected behavior.

Understanding the temporary drive on Windows Azure Virtual Machines

http://blogs.msdn.com/b/mast/archive/2013/12/07/understanding-the-temporary-drive-on-windows-azure-virtual-machines.aspx

Azure Service Management (ASM)

Here I’m going to use the new Azure Portal to create a new VM using the “classic” ASM API, please note the “Select a deployment model” combo-box in the first screen below:

Even if using “classic” old-style model, you will be prompted for a “Resource Group” and a “Virtual Network”: please keep in mind that you cannot mix “Classic” (ASM created) and “Resource Manager” (ARM created) VMs inside the same Azure VNET. Additionally, the VNET you created must be consistent with the deployment model (see print screen above) you will use for the VM. Also for VNETs, Azure Portal will present you the option to select under which model you will create your Virtual Network:

In my current example, I created a Standard_D1 VM, let’s try to see what happens if you try to change the VM size in the Azure Portal: the only options you will see as available are inside D-SERIES and A-SERIES SKUs; all the other options related to G/GS-SERIES and D/DV2/DS-SERIES will be grayed out!

The technical reason is simple: in ASM is not possible to change the Azure Host/Cluster type where your VM (and Cloud Service) has been initially created. The first VM in the Cloud Service then will determine which VM types/SKUs can be included later in the same Cloud Service. It is not possible to mix different VM SKUs in the same Cloud Service, with the exception of A-SERIES and D-SERIES, but you can use multiple Cloud Services joined to the same VNET. NOTE: Cloud Service concept is deprecated in ARM. Even if you try to stop the VM with de-allocation, that is default behavior in the Portal and PowerShell, you will not be able to change the VM SKU. If you try, for example, to add a second G-SERIES VM to an existing Cloud Service where a first A-SERIES VM has been already created, you will see the error below (from PowerShell):

 

New-AzureVM : Compute.CannotUpgradeDeploymentToNewRoleSize : Unable to upgrade the deployment. The requested VM size 'Standard_G1' may not be available in the resources supporting the existing deployment.

Please try again later, try with a different VM size or smaller number of role instances, or create a deployment under an empty hosted service with a new affinity group or no affinity group binding.

Now you may wonder why it is possible to change between A-SERIES and D-SERIES, that’s absolutely a good question and you will find the answer in my blog post below:

Azure A-SERIES, D-SERIES and G-SERIES: Consistent Performances and Size Change Considerations

http://blogs.msdn.com/b/igorpag/archive/2014/11/11/azure-a_2d00_series-and-d_2d00_series-consistent-performances-and-size-change-considerations.aspx

Long-story short: A-SERIES and D-SERIES are deployed in an Azure cluster type able to support both SKUs, that’s why you can change sizes inside both SKU families without any problem. There is an important exception that apply to A8/A9/A10/A11 VM sizes: even if formally included in the A-SERIES SKUs, these are special since “dedicated” to HPC workload with Infiniband secondary NIC (A8/A9) and faster CPU (A8/A9/A10/A11). Since Azure uses different cluster types for these VM sizes, same rules apply here as they would be in a different SKU.

Azure Resource Manager (ARM)

Now let’s try to create a new VNET and a new VM both using the “Resource Manager” deployment model. It is worth mentioning that a “Resource Group” in ARM can contain mixed “classic” and “resource manager” resources since a pure logical container, while a VNET can’t. After deployment is completed, if you try to change SKU, for example between current Standard_D1 and Standard_G3, the correspondent option will not be enabled: you will be only able to change the size inside the same SKU. Now let’s try to stop the VM with default de-allocation behavior and see what happens if you select “Size” in the “Settings” blade of the VM in the new Azure Portal:

NOTE: Some options may be still grayed out if the Azure region you selected is not offering yet the SKU you are looking for.

As you can see, you can select every VM SKU available in Azure, even GS-SERIES using Premium Storage. In this specific example, I used a single VM, but if you have more VMs tied together in the same “Availability Set”, you will have to stop all of them before attempting the resize operation across different SKUs. Conclusion: if you create a VM in Azure using ARM deployment model API, you will be able to change the SKU (and size) with no restriction as happened in ASM, the only pre-requisites is to stop (and then later restart) the VM. IMPORTANT: if you don’t stop the VM, you will be allowed only to change to sizes inside the same SKU, if you want cross-SKU migration you will have to stop the VM.

Changing VM SKU/size is pretty simple also in PowerShell (using ARM). You can use the following fragment as a starting example, all the magic happens changing the “HardwareProfile” VM property as desired:

Premium vs. Standard Storage

What about the storage? I mean, if you change from non-Premium VM SKUs, for example D-SERIES or G-SERIES, to Premium capable like DS-SERIES or GS-SERIES, what will happen? Absolutely nothing, and it is logical: if you have disks, data or OS, created on Standard storage, once the VM SKU will be changed, those disks will not move and will remain on the same non-Premium storage account used at creation time. If you want VM disks to be migrated on Premium Storage, you will have to export them, copy to Premium storage account, delete the VM retaining the disks (as safe measure), then create a new VM (DS-SERIES or GS-SERIES) using the just copied disks. On the other hand, if you have a VM created with OS, or data disks, on Premium storage account, you will not be able to downsize to non-Premium capable VM SKUs. When I tried using the Azure Portal, I got the screen below, even if the received a notification reporting successful completion of the operation:

If you use PowerShell to try this operation, you will get this error message:

An excellent article you can read on all possible scenarios and related allocation failures is reported below:

Allocation Failure and Remediation

https://azure.microsoft.com/en-us/blog/allocation-failure-and-remediation

Something similar will happen if you downsize the VM SKU to a size that is not able to support the existing number of disks: if you have a VM with 8 disks attached, you will not be able to move to a VM SKU/Size that is not able to host all of them. Of course, you can detach some of the disks and do the operation, but you will not able to attach them anymore without changing again.

Sizes for virtual machines

https://azure.microsoft.com/en-us/documentation/articles/virtual-machines-size-specs

If you want to have more information on which VM SKU is more suitable for your application workload, you can view the recording of the AzureCon 2015 session below on Channel9:

Deciding between different virtual machine sizes

https://channel9.msdn.com/Events/Microsoft-Azure/AzureCon-2015/ACON213

Conclusions

As you have seen above, ARM offers great flexibility compared to ASM deployment model and will essentially permit to move between every SKU and size, but you will need to consider which storage type you want to use. You will be eventually only required to stop your VM in
order to change SKU, not required for changing size but this will still cause a reboot. On the other hand, ASM places lots of more restrictions: this is mainly due to the Cloud Service concept and to the Azure Cluster pinning that is automatically done at first VM creation time. If you are creating your VMs today, it is highly recommended to use ARM, ASM should be limited to scenarios where you have to integrate with existing ASM deployments.

That’s all I wanted to share with you, hope you will find this content useful and interesting, feel free to post your comments and feedbacks here. Remember that you can also follow me on Twitter ( @igorpag).
Best Regards.

P.S. = Kudos to Daniele Maso from Microsoft Azure Support for reviewing this content.

Comments

  • Anonymous
    November 04, 2015
    Your article is very helpful. I think Microsoft is failing to deliver on the promise of the cloud. Why are there so many limitations. Why can't I change standard storage to premium at the flip of a switch. We're talking about layers of indirection. Our data is already being replicated all over the place. It's not sitting on one physical disk some where. Why can't it be replicated to premium storage? They need to provide an automated upgrade path. Our already created VMs shouldn't be left behind in some back water unable to use the improving features. We moved to the cloud to avoid that type of problem.

  • Anonymous
    November 04, 2015
    Hi, thank you for your feedback. Regarding limitations, in ARM essentially there are no more limitations as in ASM, the only remaining one is related to Premium vs. Standard storage. Only one left I think is not that bad, considering that on other cloud platforms there are many more based on my personal experience. But even with Standard/Premium storage, there is a very easy way to solve, that is using AZCOPY tool to manually copy disks across different storage account types. This is a server-to-server operation and then very fast. Regards.

  • Anonymous
    November 25, 2015
    Is there a way to resize the VMs when using an ARM template. If I redeploy an existing template, it doesn't change the size of the existing VMs... My idea is to quickly deploy the VMs using large size and at the end of the deployment, resize them as needed.

  • Anonymous
    November 28, 2015
    Hi JJ, Yes, it is possible, simply change the VM size and re-deploy the template, just checked and it works.

  • Anonymous
    March 02, 2016
    Igor,Is there some API or dataset that we can use to identify if an upgrade/downgrade from one VM to another is possible? If we are writing our own interface using the Azure APIs, it does not seem like we have access to all of the data needed to prevent a user from making an illegal resize request.Thanks,Patrick

    • Anonymous
      March 13, 2016
      Hi Patrick,Unfortunately not possible today. You need to know which VM size/SKU you want to change and trap eventual errors.Regards.

  • Anonymous
    March 15, 2016
    How do you resize using ARM cmdlets? I've tried using $vm.HardwareProfile.vmSize (wrapped in inlinescript) to set the size but i get the errorERROR: The property 'vmSize' cannot be found on this object. Verify that the property exists and can be set.Notes: using VmSize as suggested by ISE autocomplet dosen't work either and not wrapping it in inlinescript gives "This type of assignment is not supported"

    • Anonymous
      March 16, 2016
      Hi Rune,To resize in ARM I used the code fragment described in this post and worked perfectly, just tried also today. Don't know why didn't work in your case, sorry.
      • Anonymous
        March 16, 2016
        Any chance you can share with me exactly the code you say works? The one in this article handles Classic VM's and not ARM VM's, right? I'm trying to resize ARM VMs using the Verb-AzureRmNoun cmdlets
        • Anonymous
          March 17, 2016
          Here is....$VM = Get-AzureRmVM -ResourceGroupName $rgname -Name "igor-w1"$VM | Stop-AzureRmVM -Force$VM.HardwareProfile.VmSize = 'Standard_A2' # Changed from 'Standard_A1'$VM | Update-AzureRmVM -ResourceGroupName $rgname$VM | Start-AzureRmVM -ResourceGroupName $rgnameregards.
          • Anonymous
            March 17, 2016
            The comment has been removed
          • Anonymous
            March 18, 2016
            My pleasure Rune....