Creating Dev and Test Environments with Windows PowerShell

[アーティクル]
06/05/2015

This post will discuss creating application environments with Windows PowerShell. We will use these environments in subsequent posts.

Background

I have participated in a series of readiness workshops with our top GSI partners. Part of the workshop includes case studies where we have the participants review requirements and propose solutions within a few constraints. One of the case studies involves Visual Studio Release Management. It’s a very interesting case study, but I felt it could have been so much better with a demo. You know me, as I come up with ideas like this I try to get some time at the keyboard to build the demo and then share it here on my blog. This time is no exception.

The case study shows off Visual Studio Release Management to several environments. I’ll get to that part later, for now we will focus on designing the environment to suit our needs.

Requirements

We will have three environments: dev, stage, and prod. Each environment must have an uptime SLA of 99.5 or higher while minimizing cost. These virtual machines must be accessible from on-premises using HTTP over port 8080, but not accessible externally (no HTTP endpoint exposed). A CPU spike in one environment should not affect the other environments.

If we break the requirements down, we can see that there are three environments, each requiring an SLA of 99.95% or higher while minimizing cost. We implement two virtual machines in each environment and place them within an availability set to meet the 99.95% SLA requirement. To minimize our cost during the POC phase, we will use the Small VM size (for more information on VM sizes, see Virtual Machine and Cloud Service Sizes for Azure). The part about HTTP over port 8080, we’ll take care of that in a future post as it requires some internal configuration of the VM, but we can take care of the requirement not to expose port 8080 simply by not adding an endpoint for the environment. We’ll partially address the on-premises connectivity in this post, but that will also need a little more work to complete later. Finally, we address the customer’s concerns about CPU by placing each environment within its own cloud service.

Our proposed architecture looks like the following.

I created this environment using my MSDN subscription in about an hour just using the new Azure portal (https://portal.azure.com). However, it’s been awhile since I did anything with PowerShell, so let’s show you how to do this using the service management PowerShell SDK.

The Virtual Network

The first task is to create a virtual network XML configuration file. To be honest, I created this using the old Azure portal (https://manage.windowsazure.com), then exported the VNet to a file. In the file you can see the 3 subnets with the incredibly clever names of Subnet-1, Subnet-2, and Subnet-3. The virtual network itself has a name, in this file it’s “kirketestvnet-southcentral”. You probably want to change that Smile The value “kirketest” is part of a naming scheme that I use, I prefix all of the services (virtual network, storage, and cloud service) with the same value helping to avoid name collisions as the storage and cloud service names must be globally unique. In this example, I’ve also added a gateway. We aren’t going to create the gateway just yet, but we will leave the gateway subnet in the definition for now.

NetworkConfig.xml

<NetworkConfiguration xmlns:xsd="https://www.w3.org/2001/XMLSchema" xmlns:xsi="https://www.w3.org/2001/XMLSchema-instance" xmlns="https://schemas.microsoft.com/ServiceHosting/2011/07/NetworkConfiguration">
<VirtualNetworkConfiguration>
<VirtualNetworkSites>
<VirtualNetworkSite name="kirketestvnet-southcentral" Location="South Central US">
<AddressSpace>
<AddressPrefix>10.0.1.0/24</AddressPrefix>
</AddressSpace>
<Subnets>
<Subnet name="Subnet-1">
<AddressPrefix>10.0.1.0/27</AddressPrefix>
</Subnet>
<Subnet name="Subnet-2">
<AddressPrefix>10.0.1.32/27</AddressPrefix>
</Subnet>
<Subnet name="Subnet-3">
<AddressPrefix>10.0.1.64/26</AddressPrefix>
</Subnet>
<Subnet name="GatewaySubnet">
<AddressPrefix>10.0.1.128/29</AddressPrefix>
</Subnet>
</Subnets>
<Gateway>
<VPNClientAddressPool>
<AddressPrefix>10.0.0.0/24</AddressPrefix>
</VPNClientAddressPool>
<ConnectionsToLocalNetwork />
</Gateway>
</VirtualNetworkSite>
</VirtualNetworkSites>
</VirtualNetworkConfiguration>
</NetworkConfiguration>

To set the virtual network configuration, you use the following PowerShell script.

Set-AzureVNetConfig

$vnetConfigFilePath = "C:\temp\NetworkConfig.xml"
Set-AzureVNetConfig -ConfigurationPath $vnetConfigFilePath

A word of caution here… this will update ALL of the virtual networks for your subscription. If you use the XML as-is above, that is the same as telling Azure to delete existing virtual networks that you may have and then add the new network named “kirketest-southcentral”. Luckily, if those virtual networks are in use, the operation will fail. I find it much less risky to simply export the existing virtual network, make whatever changes I need to, then use the Set-AzureVNetConfig to apply all of the changes.

Creating Virtual Machines

When you create a virtual machine in Azure, you choose a base image. Using Get-AzureVMImage, you can get a list of all the available images and their locations. The image name will be something family-friendly like this:

a699494373c04fc0bc8f2bb1389d6106__Windows-Server-2012-Datacenter-201505.01-en.us-127GB.vhd

Notice the date and version number on that image. This image will go away at some point, replaced by a newer image. Hardcoding that image name in your script will cause you problems later, but it’s easy to just obtain the latest image version. Michael Collier provides a great post on The Case of the Latest Windows Azure VM Image with a simple solution: get the image sorted by the date it is published, descending, and get the first image. He also explains in that post how not all images are available in all locations, so you should include the location as part of your filter.

Get-LatestVMImage

function Get-LatestVMImage([string]$imageFamily, [string]$location)
{
#From https://michaelcollier.wordpress.com/2013/07/30/the-case-of-the-latest-windows-azure-vm-image/
$images = Get-AzureVMImage `
| where { $_.ImageFamily -eq $imageFamily } `
| where { $_.Location.Split(";") -contains $location} `
| Sort-Object -Descending -Property PublishedDate
return $images[0].ImageName;
}

Calling this function is really simple, just provide the location name (obtained by Get-AzureLocation).

Code Snippet

#$imageName = "a699494373c04fc0bc8f2bb1389d6106__Windows-Server-2012-Datacenter-201505.01-en.us-127GB.vhd"
$imageName = Get-LatestVMImage -imageFamily "Windows Server 2012 R2 Datacenter" -location $location

Now that we have an image, we create a new AzureVMConfig with the settings for our VM. We then create an AzureProvisioningConfig to tell the provisioning engine that this is a Windows machine with a username and password. We set a virtual network, and we are left with the configuration object. We haven’t yet told Azure to create a VM. This lets us create the configuration for multiple VMs at once before we finally start the provisioning process. Putting the VMs in an availability set is as easy as providing the –AvailabilitySetName parameter (for more information, see Michael Washam’s post, Understanding and Configuring Availability Sets).

Create Multiple VMs

New-AzureService -ServiceName $serviceName -Location $location
$vm1 = New-AzureVMConfig -Name "DEV1" -InstanceSize $size -ImageName $imageName -AvailabilitySetName $avSetName
Add-AzureProvisioningConfig -VM $vm1 -Windows -AdminUsername $adminUsername -Password $adminPassword
Set-AzureSubnet -SubnetNames $subnetName -VM $vm1
$vm2 = New-AzureVMConfig -Name "DEV2" -InstanceSize $size -ImageName $imageName -AvailabilitySetName $avSetName
Add-AzureProvisioningConfig -VM $vm2 -Windows -AdminUsername $adminUsername -Password $adminPassword
Set-AzureSubnet -SubnetNames $subnetName -VM $vm2
New-AzureVM -ServiceName $serviceName -VMs $vm1,$vm2 -VNetName $vnetName

We now have the basic building blocks out of the way. The rest of the script is simply putting everything together.

The Script

The rest of the script is fairly straightforward. We create a cloud service for each environment, two virtual machines in an availability set in each cloud service, each VM is placed within a virtual network subnet. The full script is shown here:

Full Script

function Get-LatestVMImage([string]$imageFamily, [string]$location)
{
#From https://michaelcollier.wordpress.com/2013/07/30/the-case-of-the-latest-windows-azure-vm-image/
$images = Get-AzureVMImage `
| where { $_.ImageFamily -eq $imageFamily } `
| where { $_.Location.Split(";") -contains $location} `
| Sort-Object -Descending -Property PublishedDate
return $images[0].ImageName;
}
$prefix = "mydemo"
$storageAccountName = ($prefix + "storage")
$location = "South Central US"
$vnetConfigFilePath = "C:\temp\NetworkConfig.xml"
#$imageName = "a699494373c04fc0bc8f2bb1389d6106__Windows-Server-2012-Datacenter-201505.01-en.us-127GB.vhd"
$imageName = Get-LatestVMImage -imageFamily "Windows Server 2012 R2 Datacenter" -location $location
$size = "Small"
$adminUsername = "YOUR_USERNAME_HERE"
$adminPassword = "YOUR_PASSWORD_HERE"
$vnetName = ($prefix + "vnet-southcentral")
#Use Get-AzureSubscription to find your subscription ID
$subscriptionID = "YOUR_SUBSCRIPTION_ID_HERE"
#Set the current subscription
Select-AzureSubscription -SubscriptionId $subscriptionID -Current
#Create storage account
New-AzureStorageAccount -StorageAccountName $storageAccountName -Location $location
#Set the current storage account
Set-AzureSubscription -SubscriptionId $subscriptionID -CurrentStorageAccountName $storageAccountName
#Create virtual network
Set-AzureVNetConfig -ConfigurationPath $vnetConfigFilePath
#Development environment
$avSetName = "AVSET-DEV"
$serviceName = ($prefix + "DEV")
$subnetName = "Subnet-1"
New-AzureService -ServiceName $serviceName -Location $location
$vm1 = New-AzureVMConfig -Name "DEV1" -InstanceSize $size -ImageName $imageName -AvailabilitySetName $avSetName
Add-AzureProvisioningConfig -VM $vm1 -Windows -AdminUsername $adminUsername -Password $adminPassword
Set-AzureSubnet -SubnetNames $subnetName -VM $vm1
$vm2 = New-AzureVMConfig -Name "DEV2" -InstanceSize $size -ImageName $imageName -AvailabilitySetName $avSetName
Add-AzureProvisioningConfig -VM $vm2 -Windows -AdminUsername $adminUsername -Password $adminPassword
Set-AzureSubnet -SubnetNames $subnetName -VM $vm2
New-AzureVM -ServiceName $serviceName -VMs $vm1,$vm2 -VNetName $vnetName
#Staging environment
$avSetName = "AVSET-STAGE"
$serviceName = ($prefix + "STAGE")
$subnetName = "Subnet-2"
New-AzureService -ServiceName $serviceName -Location $location
$vm1 = New-AzureVMConfig -Name "STAGE1" -InstanceSize $size -ImageName $imageName -AvailabilitySetName $avSetName
Add-AzureProvisioningConfig -VM $vm1 -Windows -AdminUsername $adminUsername -Password $adminPassword
Set-AzureSubnet -SubnetNames $subnetName -VM $vm1
$vm2 = New-AzureVMConfig -Name "STAGE2" -InstanceSize $size -ImageName $imageName -AvailabilitySetName $avSetName
Add-AzureProvisioningConfig -VM $vm2 -Windows -AdminUsername $adminUsername -Password $adminPassword
Set-AzureSubnet -SubnetNames $subnetName -VM $vm2
New-AzureVM -ServiceName $serviceName -VMs $vm1,$vm2 -VNetName $vnetName
#Production environment
$avSetName = "AVSET-PROD"
$serviceName = ($prefix + "PROD")
$subnetName = "Subnet-3"
New-AzureService -ServiceName $serviceName -Location $location
$vm1 = New-AzureVMConfig -Name "PROD1" -InstanceSize $size -ImageName $imageName -AvailabilitySetName $avSetName
Add-AzureProvisioningConfig -VM $vm1 -Windows -AdminUsername $adminUsername -Password $adminPassword
Set-AzureSubnet -SubnetNames $subnetName -VM $vm1
$vm2 = New-AzureVMConfig -Name "PROD2" -InstanceSize $size -ImageName $imageName -AvailabilitySetName $avSetName
Add-AzureProvisioningConfig -VM $vm2 -Windows -AdminUsername $adminUsername -Password $adminPassword
Set-AzureSubnet -SubnetNames $subnetName -VM $vm2
New-AzureVM -ServiceName $serviceName -VMs $vm1,$vm2 -VNetName $vnetName

The Result

I ran the script using my MSDN subscription. In about 10 minutes I had 6 virtual machines grouped within 3 cloud services and 3 virtual networks. I could then change the subscription and prefix variables then run the script in a different subscription. I could have done all of this using the portal (the first time, I did), but once I had the script completed it became an asset to create the environments in a repeatable and consistent manner.

The virtual network shows the resources that are deployed into the correct subnets.

Looking at the Configure tab of a single virtual machine lets us see that the virtual machine is part of an availability set.

Clean Up

This is my MSDN subscription, so I don’t want to leave resources lying around if I am not using them. Clean up is simple, I run another script to delete everything I just created. Again, a word of caution: this deletes the virtual machines, the associated VHD files, the cloud services, the storage account, and the virtual network configuration. This is not a reversible operation.

Clean Up

$prefix = "mydemo"
$storageAccountName = ($prefix + "storage")
$subscriptionID = "YOUR_SUBSCRIPTION_ID_HERE"
#Set up credentials
Add-AzureAccount
#Set the current subscription
Select-AzureSubscription -SubscriptionId $subscriptionID -Current
$serviceName = ($prefix + "DEV")
#The following command deletes the associated VHD, but takes awhile
Get-AzureVM -ServiceName $serviceName | %{Remove-AzureVM -Name $_.Name -ServiceName $serviceName -DeleteVHD}
Remove-AzureService -ServiceName $serviceName -Force
$serviceName = ($prefix + "STAGE")
#The following command deletes the associated VHD, but takes awhile
Get-AzureVM -ServiceName $serviceName | %{Remove-AzureVM -Name $_.Name -ServiceName $serviceName -DeleteVHD}
Remove-AzureService -ServiceName $serviceName -Force
$serviceName = ($prefix + "PROD")
#The following command deletes the associated VHD, but takes awhile
Get-AzureVM -ServiceName $serviceName | %{Remove-AzureVM -Name $_.Name -ServiceName $serviceName -DeleteVHD}
Remove-AzureService -ServiceName $serviceName -Force
#Remove storage account. This will fail if the
#disks haven't finished deleting yet.
Remove-AzureStorageAccount -StorageAccountName $storageAccountName
#Remove all unused VNets
Remove-AzureVNetConfig

Coming up next we’ll look at how we can establish on-premises connectivity for just a few devices, and then we’ll turn our attention to deploying some code and services to these virtual machines using Visual Studio Release Management.