N-series - first Azure VMs with GPU

  • Article

Great news - Azure N-series Virtual Machines are now available in public preview. N-Series instances are enabled with NVIDIA’s cutting edge GPUs to allow you to run GPU-accelerated workloads and visualize them. These powerful sizes come with the agility you have come to expect from Azure, paying per-minute of usage.

N-Series VMs are split into two categories: NC-series and NV-series. NC-Series (compute-focused GPUs) are powered by Tesla K80 GPUs and offers the fastest computational GPU available in the public cloud. Furthermore, unlike other providers, these new SKUs expose the GPUs through discreet device assignment (DDA) which results in close to bare-metal performance. You can now crunch through data much faster with CUDA across many scenarios including energy exploration applications, crash simulations, ray traced rendering, deep learning and more. The Tesla K80 delivers 4992 CUDA cores with a dual-GPU design, up to 2.91 Teraflops of double-precision and up to 8.93 Teraflops of single-precision performance.

NV-Series is focused more on visualization. Data movement has traditionally been a challenge with HPC scenarios using large datasets produced in the cloud. With the Azure NV-Series, you’ll be able to use Tesla M60 GPUs and NVIDIA GRID in Azure for desktop accelerated applications and virtual desktops. With these powerful visualization GPUs in Azure, you will be able to visualize graphic-intensive workflows to get superior graphics capability and run single precision workloads such as encoding and rendering. The Tesla M60 delivers 4096 CUDA cores in a dual-GPU design with up to 36 streams of 1080p H.264.

N-series VMs are powered by Intel Xeon E5-2690v3 CPUs. You can scale from 6 cores + 56 Gb RAM + 1x GPU (NV6/NC6) to 24 cores + 224 Gb RAM + 4x GPUs (NV24/NC24). List of all N-series VM sizes is available here. These VMs are currently available in 4 Azure Regions: East US (NC only), South Central US, West Europe and Southeast Asia. Currently you can use Standard (non-SSD)Storage only for data, but all N-series VMs include at least 340Gb of temporary SSD storage.

Prices are available here. Don't get confused with the high monthly pricing. Yes, N-series VMs are much more expensive than A-series.

capture_13122016_123439

But don't forget that you are charged on per-minute basis. Designers or engineers won't need to work with heavy graphics 24/7. So this pricing is more indicative:

capture_13122016_123426

For example, NV6 VM (Windows Server 2016, South Central US, $1.35/hour) for an engineer, that works 8 hours a day 22 days per month (176 hours total), will cost around $237 per month. Combined with a storage and network traffic cost, it will be around $250 per month per end-user. It's $9000 for 3 years, which is comparable with a professional graphics station with similar hardware characteristics ($6000-$10000), but:

  1. Hardware graphics station can't scale. It's hard to predict how much compute and graphics power an engineer or designer will need for his work. In Azure you can scale from NV6 to NV24 and back in minutes, this is just a simple reboot.
  2. Hardware graphics stations cost a lot. This is a significant CAPEX for the organization. In Azure you pay as you grow, no upfront investments required.
  3. Hardware graphics station can fail. OS can fail. You will need to find a replacement for a user. In Azure you can spin up a replacement VM in minutes.
  4. You don't pay when your engineer or designer don't use his workstation. He can be on vacation, on sick leave, or he can leave the company. It will be hard to sell the hardware graphics station, but in Azure you just turn off the VM and don't pay anymore.

Azure N-series VM is a great option when you've just started a business. Anyway, you will be able to purchase a hardware graphics station in the future when you will have more budget and you will know how much compute and graphics power you need for your employees.

Deployment of N-series

Let's test Azure N-series virtual machine in action. Login to the Azure Portal and create new VM. I will deploy NV6 VM with Windows Server 2016 in West Europe region.

capture_003_12122016_185006

VM creates in less than a minute, which is much faster than a regular A-series VM. I assume that such VMs are pre-created, and then just booted up and specialized with user settings.

Next step - install Nvidia drivers, like described here.

capture_001_12122016_181252

Reboot and check that Nvidia GPU now visible in Device Manager.

capture_002_12122016_183203

Then let's enable H.264/AVC 444 mode for RDP. I've already mentioned this new RDP feature of Windows Server 2016, that improves the picture quality and increases FPS. To do than, open Group Policy Editor (gpedit.msc) and go to Computer Configuration -> Administrative Templates -> Windows Components -> Remote Desktop Services -> Remote Desktop Session Host -> Remote Session Environment. Then enable 2 policies:

  • Prioritize H.264/AVC 444 Graphics mode for Remote Desktop connections
  • Configure H.264/AVC hardware encoding for Remote Desktop connections

capture_004_12122016_185207

Now let's test it in real-life scenario.

Testing

To test the performance of GPU inside a VM, that runs in the Azure datacenter 2200km far away from my location, I will use Nvidia demos. Lifelike Human Face Rendering demo is a good example of 3D rendering in real time.

Let's test the network latency between my location (Moscow, Russia) and VM in West Europe Azure region (Netherlands). I use GPON Internet connection, device is connected through 802.11n Wi-Fi. As you see, average latency is 50ms.

capture_13122016_120148

Let's start the demo on a full screen with FullHD resolution. It looks great!

face

I use Intel Compute Stick with Windows 10 as an endpoint. Just imagine to see such 3D quality on the $120 device, that fits in a pocket.

faceonstick

Here is the live recording from my phone. As you see, FPS is pretty stable, so such solution will be suitable for designers and engineers.

[embed]https://youtu.be/2l5Z688ym9Y[/embed]

Cost optimizations

To reduce the cost, we need to configure automatic Power-off after work hours and add capabilities to end-users to power on and power off their VMs by themselves. This can be done by granting them Contributor rights on the VM level.

capture_13122016_141436

 

Then I will put 2 scripts on the desktop. First script will power-on the users VM, second script will shut is down. These scripts leverage simple Azure RM cmdlets - Start-AzureRmVM and Stop-AzureRmVM.

capture_007_12122016_190158

You can enable automatic Power-off in the VM properties.

capture_13122016_114338

Also it will be a good practice to switch external IP to Static mode. Otherwise it will change when the VM will be booted back after de-allocation.

capture_13122016_143345

User turn on his VM, wait for 3 minutes, and then he clicks on the RDP icon and connects to his VM. When the work is done, he turns off the VM. If he'll forget to it, VM will automatically shut down at the scheduled time.

This is only the beginning of GPU-powered VMs in Azure. They open a lot of new opportunities for different industries. Let's be in touch and see how this idea will evolve after General Availability in 2017.

Comments

  • Anonymous
    March 02, 2017
    It's very good article to know about the GPU solution on azure. I have the question about the demo. In your demo you use the RDP access the virtual desktop and show 3D rendering smoothly. Is it means the RDP can support the fully 3D acceleration(Like HDX or PCOIP),right ? If not, what kind of DX and OpenGL api that RDP can support, thanks!
    • Anonymous
      March 02, 2017
      This is regular RDP 10 with AVC 444 enabled, and yes, it supports 3D acceleration. Check here for details: https://blogs.technet.microsoft.com/hybridcloudbp/2016/11/15/new-rds-capabilities-in-windows-server-2016-for-service-providers/
      • Anonymous
        March 02, 2017
        Refer the link you provided that only mentioned the RemoteFX now can support up to OpenGL 4.4. How about the regular RDP 10? Thanks!
        • Anonymous
          March 03, 2017
          RDP doesn't care if it is OpenGL or Direct3D. It just packages the 3D image into the RDP protocol flow. There is no "RemoteFX or RDP". RDP is a transport protocol, RemoteFX is a virtual GPU technology with API redirection.Some of the RDP features, that are valuable for 3D image (like AVC 444) are enabled by default only if RemoteFX is used, but they can be enabled manually and used even without RemoteFX.In the described case RemoteFX wasn't used, because it is Discrete Device Assignment (GPU pass-through into the VM). So I enabled AVC 444 manually to increase 3D quality inside RDP.
  • Anonymous
    March 19, 2017
    GPUs on Azure. This is cool, thanks for posting about it Kirill!I attempted to try this out using a Mac laptop running beta Remote Desktop client.I suspect the Mac client does not support AVC444 because of low frame rate (5-8 fps).Is there someone on Apple Productivity Experience Group who could please help us learn if Mac AVC444 support is on the roadmap? tried beta Mac Remote Desktop client at https://rink.hockeyapp.net/apps/5e0c144289a51fca2d3bfa39ce7f2b06/RDC 8.2.31macOS 10.12.3.2
  • Anonymous
    March 25, 2017
    It's difficult to find experienced people on this topic, but you seem like you know what you're talking about! Thanks
  • Anonymous
    May 08, 2017
    What is the FPS that you can get max with Tesla K80 - 1 Dies with these settings on RDP?
  • Anonymous
    June 28, 2017
    Hi!Can I launch DirectX applications from RDP connection?I've read your article, and nVidia demos work, but DirectX apps don't work...Do I need to configure something?
    • Anonymous
      June 28, 2017
      This is a technology, that works on the guest OS level, RDP won't care which API was used to generate 3D picture. So it should be fine.
  • Anonymous
    August 03, 2017
    Thanks for the great article, I follow your article and successfully created NC and NV VMs.But my problem is, there is no vRam. I installed Photoshop and it gives error that system vRam is less than 512MB. I tried to find out my vRam but was unsuccessful.. Any solution?