Common questions about Azure VMware Solution

This article answers commonly asked questions about Azure VMware Solution.

General

What is Azure VMware Solution?

As enterprises pursue IT modernization strategies to improve business agility, reduce costs, and accelerate innovation, hybrid cloud platforms are key enablers of customers' digital transformation. Azure VMware Solution combines VMware's Software-Defined Data Center (SDDC) software with Microsoft's Azure global cloud service ecosystem. In addition, Azure VMware Solution meets performance, availability, security, and compliance requirements. For more information, see What is Azure VMware Solution.

Where is Azure VMware Solution available today?

The service is continuously being added to new regions. For details, see the latest service availability information.

Who supports Azure VMware Solution?

Microsoft delivers support for Azure VMware Solution. You can submit a support request. For Cloud Solution Provider (CSP) managed subscriptions, the first level of support provides the Solution Provider in the same fashion as CSP does for other Azure services.

Can workloads running in an Azure VMware Solution instance integrate with Azure services?

All Azure services are available to Azure VMware Solution customers. Performance and availability limitations for specific services should be addressed on a case-by-case basis.

What guest operating systems are compatible with Azure VMware Solution?

You can find information about guest operating system compatibility with vSphere by using the VMware Compatibility Guide. To identify the version of vSphere running in Azure VMware Solution, see VMware software versions.

What does the change control process look like?

Updates made follow Microsoft Azure's standard change management process. Customers are responsible for any workload administration tasks and the associated change management processes.

How is this version different from Azure VMware Solution by CloudSimple?

With the new Azure VMware Solution, Microsoft and VMware have a direct cloud provider partnership. Microsoft designed, built, and supported the new solution. The solution is endorsed by VMware. Architecturally, the solutions are consistent, with the VMware technology stack running on a dedicated Azure infrastructure.

Billing

Is there a Service Level Agreement (SLA) on disk replacement when failures occur?

Any hosts with disk issues are replaced. It rolls up to 99.9 SLA availability of the Azure VMware Solution service.

How is pricing structured for Azure VMware Solution?

For general questions on pricing, see the Azure VMware Solution pricing page.

Is VMware HCX Enterprise available, and if so, how much does it cost?

VMware HCX Enterprise is available on Azure VMware Solution at no other cost and is enabled by default.

Will traffic between on-premises and Azure VMware Solution over ExpressRoute incur any outbound data transfer charge in the metered data plan?

Traffic in the Azure VMware Solution ExpressRoute circuit isn't metered. No billing for any Azure VMware Solution ExpressRoute circuit, or for Global Reach charges between Azure VMware Solution private clouds. This scenario includes Azure VMware Solution to on-premises, other than standard egress charges for traffic from your Azure ExpressRoute circuit connection to your on-premises site from Azure. These fees are charged according to Azure ExpressRoute pricing plans with the Metered Billing Model. If you're using the Azure ExpressRoute Unlimited Billing Model, egress traffic isn't charged.

  • Azure VMware Solution to Azure Virtual Network is through an internal ExpressRoute circuit and is free of cost, regardless of region location (same region or cross-region).

  • Azure VMware Solution to on-premises site is done through Azure Virtual Network or ExpressRoute Global Reach (between the internal ExpressRoute and external ExpressRoute). It's still free aside from the standard egress charges (Metered Billing Model) from the ExpressRoute to on-premises network. For the Unlimited Billing Model, there are no data charges.

For example:

  • If we connect an Azure Virtual Network in Azure West Europe to an Azure VMware Solution private cloud in West Europe, there are no ExpressRoute charges other than the ExpressRoute gateway charges.

  • If we connect an Azure Virtual Network in Azure North Europe to an Azure VMware Solution private cloud in West Europe, there are no ExpressRoute charges other than the ExpressRoute gateway charges.

  • If you connect an Azure VMware Solution private cloud in West Europe to an Azure VMware Solution private cloud in North Europe via ExpressRoute Global Reach. There are no ExpressRoute Global Reach data transfer (egress and ingress) charges. There are charges when using an ExpressRoute gateway.

Diagram showing Azure VMware Solution data transfer charges.

Is it necessary to procure other VMware licensing and resources other than the AV36 instance when migrating from the on-premises VM environment with an L2 extension?

No, you don't need to procure other VMware licensing beyond the Azure VMware Solution service. For more information, see the Azure VMware Solution pricing page to see what VMware technology is included.

Support

How do I request a host quota increase for Azure VMware Solution?

Whether you want more hosts for an existing private cloud or you're creating a new private cloud, you need to submit a support ticket to have your hosts allocated. For more information, see Request host quota for Azure VMware Solution.

What accounts do I need to create an Azure VMware Solution private cloud?

You need an Azure account in an Azure subscription.

Are Red Hat solutions supported on Azure VMware Solution?

Microsoft and Red Hat share an integrated, colocated support team that provides a unified contact point for Red Hat ecosystems running on the Azure platform. Like other Azure platform services that work with Red Hat Enterprise Linux, Azure VMware Solution falls under the Cloud Access and integrated support umbrella. Red Hat Enterprise Linux supports running on top of Azure VMware Solution within Azure.

Customer communication

How can I receive an alert when Azure sends service health notifications to my Azure subscription?

You can find service issues, planned maintenance, health advisories, and security advisories notifications published through Service Health in the Azure portal. You can take timely actions when you set up activity log alerts for these notifications. For more information, see Create Service Health alerts using the Azure portal.

Screenshot of Service Health notifications.

Configuration and setup

How long does it take to provision the initial three hosts in a cluster?

At the moment, the provisioning can take roughly 3-4 hours. Adding a single node in existing/same cluster takes between 30 - 45 minutes.

Can I use the folder name "AVS-vendor-folders" for vCenter Server VM folders in Azure VMware Solution?

No, "AVS-vendor-folders" is a reserved name within Azure VMware Solution and using it might lead to conflicts with the intended functionality or cause unexpected behavior within the vCenter management environment specific to Azure VMware Solution. Choose an alternative folder name that aligns with your organizational needs while avoiding conflicts with predefined naming conventions in Azure VMware Solution.

VMware solution software

Can Azure VMware Solution VMs be managed by VMRC?

Yes. Provided the system it's installed on can access the private cloud vCenter Server and is using public DNS to resolve ESXi hostnames.

Are there special instructions for installing and using VMRC with Azure VMware Solution VMs?

No. To meet the VM prerequisites, follow the instructions provided by VMware.

Can I use vRealize Suite running on-premises?

vRealize Automation, vRealize Operations Manager, and vRealize Network Insight are certified for use with Azure VMware Solution when those products are installed in an on-premises data center. The cloud-based versions of these products--vRealize Automation Cloud, vRealize Operations Cloud, and vRealize Network Insight Cloud--are also certified for use. Installing these products within an Azure VMware Solution private cloud isn't supported.

Can I migrate vSphere VMs from on-premises environments to Azure VMware Solution private clouds?

Yes. This is possible and recommended via the VMware HCX add-on. While cross vCenter cold migration and cloning is possible, not cross vCenter vMotion, HCX is the fully-supported method.

Is a specific version of vSphere required in on-premises environments?

The on-premises environment must be running vSphere 6.5 or later if VMware HCX will be used to migrate VMs.

How do I migrate a VM to a different plan?

To migrate an Azure Virtual Machine (VM) to a different plan, follow these steps:

  1. Stop the VM in the Azure portal by selecting your VM and then selecting 'Stop' to deallocate resources.
  2. With the VM stopped, access the 'Size' setting of the VM.
  3. In the 'Choose a size' panel, select a new size that is compatible with either the current or desired series.
  4. Select the 'Resize' button to apply the size change.
  5. Restart the VM to finalize the migration.

Remember that you can only resize a VM within the same series or to an available series in the same Azure region. Ensure the new plan supports your VM's storage and networking configurations.

Is VMware HCX supported on VPNs?

Yes, provided VMware HCX Network Underlay Minimum Requirements are met.

What versions of VMware software are used in private clouds?

The VMware solution software versions used in new deployments of Azure VMware Solution private clouds are:

Software Version
VMware vCenter Server 7.0 U3o
VMware ESXi 7.0 U3o with TianfuCup HotPatch
VMware vSAN 7.0 U3
VMware vSAN on-disk format 15
VMware vSAN storage architecture OSA
VMware NSX 4.1.1
VMware HCX 4.8.2
VMware Site Recovery Manager 8.7.0.3
VMware vSphere Replication 8.7.0.3

The current running software version is applied to new clusters added to an existing private cloud.

How often is the VMware solution software (ESXi, vCenter Server, NSX) patched, updated, or upgraded in the Azure VMware Solution private cloud?

One benefit of Azure VMware Solution private clouds is that the platform is maintained for you. Microsoft is responsible for the lifecycle management of VMware software (ESXi, vCenter Server, and vSAN) and NSX appliances. Microsoft is also responsible for bootstrapping the network configuration, like creating the Tier-0 gateway and enabling North-South routing. You’re responsible for the NSX SDN configuration: network segments, distributed firewall rules, Tier 1 gateways, and load balancers.

Note

A T0 gateway is created and configured as part of a private cloud deployment. Any modification to that logical router or the NSX edge node VMs could affect connectivity to your private cloud and should be avoided.

Microsoft is responsible for applying any patches, updates, or upgrades to ESXi, vCenter Server, vSAN, and NSX in your private cloud. The impact of patches, updates, and upgrades on ESXi, vCenter Server, and NSX has the following considerations:

  • ESXi - There's no impact to workloads running in your private cloud. Access to vCenter Server and NSX isn't blocked during this time. During this time, we recommend you don't plan other activities like: scaling up private cloud, scheduling or initiating active HCX migrations, making HCX configuration changes, and so on, in your private cloud.

  • vCenter Server - There's no impact to workloads running in your private cloud. During this time, vCenter Server is unavailable and you can't manage VMs (stop, start, create, or delete). We recommend you don't plan other activities like scaling up private cloud, creating new networks, and so on, in your private cloud. When you use VMware Site Recovery Manager or vSphere Replication user interfaces, we recommend you don't do either of the actions: configure vSphere Replication, and configure or execute site recovery plans during the vCenter Server upgrade.

  • NSX - The workload is impacted. When a particular host is being upgraded, the VMs on that host might lose connectivity from 2 seconds to 1 minute with any of the following symptoms:

    • Ping errors

    • Packet loss

    • Error messages (for example, Destination Host Unreachable and Net unreachable)

    During this upgrade window, all access to the NSX management plane is blocked. You can't make configuration changes to the NSX environment for the duration. Your workloads continue to run as normal, subject to the upgrade impact previously detailed.

    During the upgrade time, we recommend you don't plan other activities like, scaling up private cloud, and so on, in your private cloud. Other activities can prevent the upgrade from starting or could have adverse impacts on the upgrade and the environment.

You're notified through Azure Service Health that includes the timeline of the upgrade. This notification also provides details on the upgraded component, its effect on workloads, private cloud access, and other Azure services. You can reschedule an upgrade as needed.

Software updates include:

  • Patches - Security patches or bug fixes released by VMware

  • Updates - Minor version change of a VMware stack component

  • Upgrades - Major version change of a VMware stack component

Note

Microsoft tests a critical security patch as soon as it becomes available from VMware.

Documented VMware workarounds are implemented in lieu of installing a corresponding patch until the next scheduled updates are deployed.

Do private clouds use VMware NSX? If so, which version is supported?

Yes, NSX is the only supported version of VMware network virtualization software.

VMware NSX 4.1.1 is used for the software-defined networking in Azure VMware Solution private clouds.

Is VMware NSX required in on-premises environments or networks that connect to a private cloud?

No, you aren't required to use VMware NSX on-premises. VMware HCX provides the necessary connectivity between on-premises vSphere and Azure VMware Solution.

Is NSX Identity Firewall and distributed IDS/IPS supported with Azure VMware Solution?

Yes with an add-on license. Azure VMware Solution supports distributed IDFW and distributed IDS/IPS with add-on firewall license. This add-on license needs to be bought from VMware and Microsoft will apply that license to the Azure VMware Solution private cloud with a support request.

Is NSX Service Insertion supported with Azure VMware Solution?

No.

Is VMware Horizon 8 2012 compatible with Azure VMware Solution?

Yes.

Migrate

Why can't I see my Sentinel Management tab in the HCX Manager when using the Sentinel Appliance service?

The Sentinel Management tab provides you access to download the Sentinel software. It appears in the HCX Interconnect interface when an HCX Enterprise license is activated, and you have deployed a service mesh with a Sentinel Gateway (SGW) and Sentinel Data Receiver (SDR) pair deployed. Also, in traditional on-premises to cloud deployments, the Sentinel tab is only visible in the Connector, not cloud manager.

If we migrate a VM created with thick provisioning on the on-premises side to Azure VMware Solution, will the VM remain thick?

You can specify the type of format you want when you migrate a VM to Azure VMware Solution. However, vSAN is primarily the datastore you'll use in Azure VMware Solution, so it depends on the storage policy that's selected. The datastore default storage policy is the RAID-1 FTT-1, which is thin provisioned. You can use Run commands to change the default datastore storage policy.

Compute

What are the CPU specifications in each type of host?

The AV36 SKU servers have dual 18 core 2.3 GHz Intel CPUs. AV36P SKU servers have dual 18 core 2.6 GHz Intel CPUs and AV52 SKU servers have dual 26 core 2.7 GHz Intel CPUs.

How much memory is in each host?

The AV36 SKU servers have 576 GB of RAM. AV36P SKU servers have 768 GB of RAM and AV52 SKU servers have 1,536 GB of RAM.

Does Azure VMware Solution support running ESXi as a nested virtualization solution?

No. VMware doesn't officially support nested virtualization.

Backup/restore

What independent software vendors (ISVs) backup solutions work with Azure VMware Solution?

Commvault, Veritas, and Veeam have extended their backup solutions to work with Azure VMware Solution. However, any backup solution that uses VMware vStorage API for Data Protection (VADP) with the HotAdd transport mode works out of the box on Azure VMware Solution. For more information, see Backup solutions for Azure VMware Solution VMs.

What about support for ISV backup solutions?

As these backup solutions are installed and managed by customers, they can reach out to the respective ISV for support.

Networking and interconnectivity

Can Azure Bastion be used for connecting to Azure VMware Solution VMs?

Azure Bastion is the service recommended to connect to the jump box to prevent exposing Azure VMware Solution to the internet. You can't use Azure Bastion to connect to Azure VMware Solution VMs since they aren't Azure IaaS objects.

How much network bandwidth is available in each ESXi host?

Each ESXi host in Azure VMware Solution is configured with four 25-Gbps NICs, two NICs provisioned for ESXi system traffic, and two NICs provisioned for workload traffic.

Are the SNMP infrastructure logs shared?

No.

Does ExpressRoute support packets exceeding MTU of 1500?

No.

Can Azure Load Balancer internal be used for Azure VMware Solution VMs?

No. Azure Load Balancer internal-only supports Azure IaaS VMs. Azure Load Balancer doesn't support IP-based backend pools; only Azure VMs or Virtual Machine Scale Set objects in which Azure VMware Solution VMs aren't Azure objects.

Can an existing ExpressRoute Gateway be used to connect to Azure VMware Solution?

Yes. Use an existing ExpressRoute Gateway to connect to Azure VMware Solution as long as it doesn't exceed the limit of four ExpressRoute circuits per virtual network. To access Azure VMware Solution from on-premises through ExpressRoute, you must have ExpressRoute Global Reach since the ExpressRoute Gateway doesn't provide transitive routing between its connected circuits.

What network IP address planning is required to incorporate private clouds with on-premises environments?

A private network /22 address space is required to deploy an Azure VMware Solution private cloud. This private address space shouldn't overlap with other virtual networks in a subscription or with on-premises networks.

How do I connect from on-premises environments to an Azure VMware Solution private cloud?

You can connect to the service in one of two methods:

  • With a VM or application gateway deployed on an Azure virtual network that is peered through ExpressRoute to the private cloud.
  • Through ExpressRoute Global Reach from your on-premises data center to an Azure ExpressRoute circuit.

How do I connect a workload VM to the internet or an Azure service endpoint?

In the Azure portal, enable internet connectivity for a private cloud. With NSX Manager, create an NSX T1 gateway and a logical switch. You then use vCenter Server to deploy a VM on the network segment defined by the logical switch. That VM has network access to the internet and Azure services.

Note

A T0 gateway is created and configured as part of a private cloud deployment. Any modification to that logical router or the NSX edge node VMs could affect connectivity to your private cloud and should be avoided.

Do I need to restrict access from the internet to VMs on logical networks in a private cloud?

No. Network traffic inbound from the internet directly to private clouds isn't allowed by default. However, you're able to expose Azure VMware Solution VMs to the internet through the Public IP option in your Azure portal for your Azure VMware Solution private cloud.

Do I need to restrict internet access from VMs on logical networks to the internet?

Yes. You need to use NSX Manager to create a firewall to restrict VM access to the internet.

Which IP range can be used for DNS service IP and DHCP server IP?

The IP address range shouldn't overlap with the IP range used in other virtual networks in your subscription and on-premises networks.

Can Azure VMware Solution use Azure Virtual WAN hosted ExpressRoute Gateways?

Yes.

Can transit connectivity be established between on-premises and Azure VMware Solution through Azure Virtual WAN over ExpressRoute Global Reach?

Azure Virtual WAN doesn't provide transitive routing between two connected ExpressRoute circuits and nonvirtual WAN ExpressRoute Gateway. ExpressRoute Global Reach allows connectivity between on-premises and Azure VMware Solution but goes through Microsoft's global network instead of the Virtual WAN Hub.

Is Windows 2008 supported as an Active Directory (AD) server or Remote Desktop Session Host (RDSH) OS in NSX?

No.

Why can't I reach the Azure VMware Solution vCenter Server Appliance and NSX Manager from on-premises or Azure Virtual Network?

By design, you won't be able to reach NSX Manager and vCenter Server Appliance (vCSA) from on-premises when only 0.0.0.0/0 (default route) is being advertised over ExpressRoute Global Reach between Azure VMware Solution and your on-premises ExpressRoute or through Azure Virtual Network to Azure VMware Solution. You need to advertise specific networking routes/subnets to access NSX Manager and vCSA.

Storage

What is the correct storage policy for the deduplication setup?

Use the thin_provision storage policy for your VM template.

What is the storage capacity of each host?

Each ESXi host has two vSAN disk groups with a capacity tier of 15.2 TB and a 3.2-TB NVMe cache tier (1.6 TB in each disk group).

Is data stored on the vSAN datastores encrypted at rest?

Yes, vSAN datastores use data-at-rest encryption by default using keys stored in Azure Key Vault. The encryption solution is KMS-based and supports vCenter Server operations for key management. When a host is removed from a vSphere cluster, data on disk is invalidated immediately.

Can I rename a datastore or cluster during creation?

No, you can't change the name of datastores or clusters.

What is the Fault tolerance of hardware failure on the vSAN?

RAID-1, FTT-1, with Object Space reservation set to Thin Provisioning is the Default Storage policy for the software-defined datacenters (SDDCs).

What is the difference between thick provisioning and thin provisioning?

Thick provisioning is reserved or preallocated storage space. Thick provisioning protects systems by allowing them to function even if the vSAN datastore is full because the space is already reserved. For example, suppose you create a 10-GB virtual disk with thick provisioning. In that case, the full amount of virtual disk storage capacity is preallocated on the physical storage where the virtual disk is created and consumes all the space allocated to it in the datastore. It won't allow other VMs to share the space from the datastore. A thin-provisioned virtual disk consumes the space that it needs initially and grows to the data space demand used in the datastore.

How many disks can fail on the vSAN before data loss occurs?

It depends on how you plan your application workloads to run inside the SDDC (private cloud). Microsoft governs these failures regularly and replaces the hardware when such events are detected from an infrastructure perspective. As a default, a setting of FTT-1 is used, which accommodates a single host's failure.

What kind of alerts can I expect to see for vSAN?

Microsoft provides alerts when capacity consumption exceeds 75%. Alternatively, you can also monitor capacity consumption metrics that are integrated into Azure Monitor.

How many 1.6-TB NVMe drives make up the disk groups to provide the 15.4 TB of raw SSD storage per host?

The AV36 SKU includes two 1.6-TB NVMe Cache and eight 1.9-TB raw storage capacity. These are then split into two disk groups. Check the AV36P and AV52 SKUs for their hardware specifications.

What is the RAID configuration of the disk groups?

The disk groups aren't RAID configured. Instead, they're just a bunch of disks (JBOD) and are directly controlled by vSAN.

Hosts, clusters, and private clouds

Is there more than one type of host available?

No. There's only one type available.

Do I use the same tools that I use now to manage private cloud resources?

Yes. The Azure portal is used for deployment and several management operations. vCenter Server and NSX Manager are used to manage vSphere and NSX resources.

Can I manage a private cloud with my on-premises vCenter Server?

At launch, Azure VMware Solution won't support a single management experience across on-premises and private cloud environments. You manage private cloud clusters with vCenter Server and NSX Manager local to a private cloud.

If a cluster is scaled up, and then workload demand falls, can it be scaled back down?

Yes, as long as you have the quota allocated against your private cloud, you can scale out your clusters. When workload demand falls, you can delete hosts from the cluster to scale it down. You can do this through the Azure VMware Solution portal.

Is the underlying infrastructure shared?

No, private cloud hosts and clusters are dedicated and securely erased before and after use.

What are the minimum and the maximum number of hosts per cluster? Can I scale my private cloud clusters?

Clusters can scale between three (minimum) and 16 (maximum) ESXi hosts.

Identity management

What accounts and privileges will I get with my new Azure VMware Solution private cloud?

You're provided credentials for a cloud admin user in vCenter Server and admin access on NSX Manager. You can also use a CloudAdmin group to incorporate Microsoft Entra ID. For more information, see Access and identity architecture.

Can have administrator access to ESXi hosts?

No, administrator access to ESXi is restricted to meet the security requirements of the solution.

What privileges and permissions will I have in vCenter Server?

You have CloudAdmin role privileges. For more information, see Access and identity architecture.

What privileges and permissions will I have on the NSX Manager?

You have CloudAdmin role privileges. For more information, see Access and identity architecture.

Note

A T0 gateway is created and configured as part of a private cloud deployment. Any modification to that logical router or the NSX edge node VMs could affect connectivity to your private cloud and should be avoided.

How can I change my credentials?

For information on resetting your credentials, see Rotate the cloudadmin credentials for Azure VMware Solution.

Are the cloudadmin extension privileges supported by Azure VMware Solution?

No. We currently don't support cloudadmin extension privileges and have no plans to support it.

CSP and multi-tenancy

Does Azure VMware Solution provide an option for hoster partners to resell the service?

Yes. For more information, see Request host quota for Azure VMware Solution.

Does Azure VMware Solution offer multi-tenancy for hosting CSP partners?

No. Currently, Azure VMware Solution doesn't offer multi-tenancy.

Does Azure VMware Solution enable a hoster partner to partition resources within the private cloud (SDDC) to manage for customers in a multi-tenanted way?

No, an Azure VMware Solution private cloud can't be shared between end customers.

I use Azure VMware Solution to create end-user applications or workloads accessed on multiple VMs through public IP. Can I sell this solution to multiple tenants?

Customers can create multitenant environments in their Azure VMware Solution private cloud and sell to customers provided the product isn't a standard VM and have added substantial intellectual property embedded in the VM as an application.

As a service provider or customer, can I use my existing VMware Cloud Director (vCD) or new vCD deployment in Azure VMware Solution?

VMware doesn't allow vCD in the cloud or from on-premises to be licensed with any Hyperscaler, including Microsoft.

Can I connect VMware Cloud Director Service (CDS) to my Azure VMware Solution instance in Azure?

Yes. You can connect your Azure VMware Solution private cloud to VMware Cloud Director Service from VMware. This integration of both services is currently in public preview.

Can Azure VMware Solution be purchased through a Microsoft CSP?

Yes, customers can deploy Azure VMware Solution within an Azure subscription managed by a CSP.

Are Reserved Instances available for purchasing through the CSP program?

Yes. CSPs can purchase reserved instances for their customers. For more information, see Save costs with a reserved instance.