XAML and Direct3D apps for Windows Phone 8

[ This article is for Windows Phone 8 developers. If you’re developing for Windows 10, see the latest documentation. ]

This topic describes the structure of a XAML and Direct3D app, and walks through the project template that’s included in Windows Phone SDK 8.0. This type of app uses the DrawingSurface control which allows you to use Direct3D to render graphics that are displayed behind or inline with XAML controls and content. The size and layout of your DrawingSurface are handled just as they are with other XAML controls.

A different app type is the Direct3D with XAML app which uses theDrawingSurfaceBackgroundGrid control. With that control, your graphics are displayed across the entire screen, below any other XAML elements on the page, including any elements in the frame. For info about choosing the control that’s right for your app, see Choosing the right project template for your game for Windows Phone 8.

Structure of a XAML and Direct3D app

A XAML and Direct3D app is made up of two components: a XAML-based Windows Phone app and a component that’s based on the Windows Phone Runtime. The Windows Phone Runtime component creates a Direct3D graphics device, which it uses to create a Texture that is shared with the XAML app. The Windows Phone Runtime component uses Direct3D to draw onto the Texture. The XAML engine maintains a swap chain that copies the texture to the screen within the bounds of the DrawingSurface control. The shared texture is referred to as a “synchronized texture” because it’s important that the XAML engine and the Windows Phone Runtime component do not operate on the texture at the same time. Coordinating all of these pieces is a complex operation, but fortunately, Windows Phone SDK 8.0 provides a project template that implements the low-level infrastructure for you. You can create a new app using this project template, then press F5 and immediately see a 3-D cube rendered to the screen of your device or Windows Phone Emulator. The remainder of this topic walks you through the project template for XAML and Direct3D apps.

Updating the XAML and Direct3D app project template

If your app uses the XAML and Direct3D project template and contains more than one page, you should make the following changes to the files included in the project template. This will ensure that memory is managed correctly as the user navigates between pages. These changes will be included in future releases of the project template.

To update the project template

1. In MainPage.xaml.cs replace the line

   Private D3DInterop As Direct3DInterop = New Direct3DInterop()
   

   private Direct3DInterop m_d3dInterop = new Direct3DInterop();
   

   with

   Private D3DInterop As Direct3DInterop = Nothing
   

   private Direct3DInterop m_d3dInterop = null;
   

2. In MainPage.xaml.cs, replace the Loaded event handler for the DrawingSurface control with the following:

   Private Sub DrawingSurface_Loaded(sender As Object, e As RoutedEventArgs)
       If D3DInterop Is Nothing Then
           D3DInterop = New Direct3DInterop()
   
           ' Set window bounds in dips
           D3DInterop.WindowBounds = New Windows.Foundation.Size(
               DrawingSurface.ActualWidth,
               DrawingSurface.ActualHeight
               )
   
           ' Set native resolution in pixels
           D3DInterop.NativeResolution = New Windows.Foundation.Size(
               Math.Floor(DrawingSurface.ActualWidth * Application.Current.Host.Content.ScaleFactor / 100.0F + 0.5F),
               Math.Floor(DrawingSurface.ActualHeight * Application.Current.Host.Content.ScaleFactor / 100.0F + 0.5F)
               )
   
           ' Set render resolution to the full native resolution
           D3DInterop.RenderResolution = D3DInterop.NativeResolution
   
           ' Hook-up native component to DrawingSurface
           DrawingSurface.SetContentProvider(D3DInterop.CreateContentProvider())
           DrawingSurface.SetManipulationHandler(D3DInterop)
       End If
   End Sub
   

   private void DrawingSurface_Loaded(object sender, RoutedEventArgs e)
   {
       if (m_d3dInterop == null)
       {
           m_d3dInterop = new Direct3DInterop();
   
           // Set window bounds in dips
           m_d3dInterop.WindowBounds = new Windows.Foundation.Size(
               (float)DrawingSurface.ActualWidth,
               (float)DrawingSurface.ActualHeight
               );
   
           // Set native resolution in pixels
           m_d3dInterop.NativeResolution = new Windows.Foundation.Size(
               (float)Math.Floor(DrawingSurface.ActualWidth * Application.Current.Host.Content.ScaleFactor / 100.0f + 0.5f),
               (float)Math.Floor(DrawingSurface.ActualHeight * Application.Current.Host.Content.ScaleFactor / 100.0f + 0.5f)
               );
   
           // Set render resolution to the full native resolution
           m_d3dInterop.RenderResolution = m_d3dInterop.NativeResolution;
   
           // Hook-up native component to DrawingSurface
           DrawingSurface.SetContentProvider(m_d3dInterop.CreateContentProvider());
           DrawingSurface.SetManipulationHandler(m_d3dInterop);
       }
   }
   

3. In Direct3DInterop.cpp, replace the definition of the CreateContentProvider method with the following:

   IDrawingSurfaceContentProvider^ Direct3DInterop::CreateContentProvider()
   {
       ComPtr<Direct3DContentProvider> provider = Make<Direct3DContentProvider>(this);
       return reinterpret_cast<IDrawingSurfaceContentProvider^>(provider.Get());
   }
   

The XAML and Direct3D app project template

This section walks you through the files included in the XAML and Direct3D app project template.

If you’re going to create a XAML and Direct3D app, you should start with the XAML and Direct3D app project template that’s included with Windows Phone SDK 8.0. To create a new project, in Visual Studio, on the File menu, click New Project, and then, under Visual C# or Visual Basic, click Windows Phone XAML and Direct3D App.

This template creates a new solution that has two projects: a managed XAML-based app and a Windows Phone Runtime component. The XAML app is almost identical to a regular Windows Phone app. In this app you can use XAML and managed code to implement a UI in the same way you would in an app that doesn’t use any Direct3D graphics. The only difference is that a few extra lines of code are added to the project that add a DrawingSurface control in which the Direct3D graphics will be displayed and sets up the connection to the Windows Phone Runtime component.

The Windows Phone Runtime component project includes some files that are similar to those in the pure native Direct3D app for Windows Phone, and similar to the Direct3D project template for Windows 8. These files handle setting up the graphics device for you, and have some example code that draws a cube to the screen. There are some additional files that are used to communicate between the Windows Phone Runtime component and the main application, as well as the underlying XAML engine that composites the Direct3D graphics into the XAML UI. If you’re anxious to get started building your app and aren’t interested in the low-level technical details that allow these apps to function, you can just read about the MainPage class, which hosts your XAML-based UI and your managed code, the CubeRenderer class, which is where the actual drawing to the shared texture is performed, and the section of the Direct3DInterop that discusses touch input. The rest of this topic is designed to give a more complete explanation of the code that’s provided in the template.

The managed XAML app

As mentioned earlier, the XAML app project that’s created as part of the XAML and Direct3D template is almost identical to a regular Windows Phone app that doesn’t use any Direct3D. However, there’s a little extra code in two of the files.

MainPage.xaml

This file is where XAML-based Windows Phone apps define their UI. With this project template, a DrawingSurface control is added for you. You can work with this control just like the other XAML controls supported on the phone. You can resize the control. You can add other controls before or after it, or use containers like StackPanel or Grid to control where it’s displayed in the UI.

<Grid x:Name="LayoutRoot" Background="Transparent">
    <DrawingSurface x:Name="DrawingSurface" Loaded="DrawingSurface_Loaded" />
</Grid>

MainPage.xaml.cs

The code-behind file for the main page is where the DrawingSurface control is hooked up to the Windows Phone Runtime component that does the actual Direct3D drawing. At the top of the page, a using directive references the namespace that contains the Windows Phone Runtime component. The namespace name is the name you provided when you created the project, appended with “Comp”. In this example, the project name is “XAMLD3DExample”.

using XAMLD3DExampleComp;

Next, a member variable of type Direct3DInterop is declared. Direct3DInterop is the class exposed by the Windows Phone Runtime component. This class will be discussed further later in this walkthrough.

Private D3DInterop As Direct3DInterop = Nothing

private Direct3DInterop m_d3dInterop = null;

The only method that the template creates in MainPage.xaml.cs is the Loaded event handler for the DrawingSurface control. The first part of this method sets the WindowBounds, NativeResolution, and RenderResolution properties of the Direct3DInterop class. These properties allow the Windows Phone Runtime component to create the synchronized texture with the same dimensions as the DrawingSurface control. The WindowBounds property uses dimensions expressed in Device Independent Pixels (dips), which is the same units that the DrawingSurface, and all controls, use for their ActualWidth and ActualHeight properties. The NativeResolution and RenderResolution properties expect dimensions expressed in physical pixels, so the size is converted by multiplying by the ScaleFactor property divided by 100. This ensures that the size, in pixels, is correct for all supported screen resolutions. The addition of .5 pixels to the formula and the Floor function call rounds up the dimensions to the nearest integer.

After setting the size properties, the SetContentProvider(Object) method of the DrawingSurface control is called with the object returned by the Direct3DInterop method, CreateContentProvider. This call sets the Windows Phone Runtime component as the provider that will use Direct3D to draw the content for the DrawingSurface control. As you’ll see later in this walkthrough, the project template is structured so that it uses the CreateContentProvider helper method to help encapsulate the low-level calls to set up the content provider.

Next, SetManipulationHandler(Object) is called and the Direct3DInterop object is passed in. This registers the Direct3DInterop to receive the PointerPressed()()(), PointerMoved()()(), and PointerReleased()()() touch events when the user touches within the DrawingSurface control. You’ll have better performance using these events directly in your Windows Phone Runtime component than you would by getting touch events in the MainPage code-behind page and then manually passing that data to the component.

Private Sub DrawingSurface_Loaded(sender As Object, e As RoutedEventArgs)
    If D3DInterop Is Nothing Then
        D3DInterop = New Direct3DInterop()

        ' Set window bounds in dips
        D3DInterop.WindowBounds = New Windows.Foundation.Size(
            DrawingSurface.ActualWidth,
            DrawingSurface.ActualHeight
            )

        ' Set native resolution in pixels
        D3DInterop.NativeResolution = New Windows.Foundation.Size(
            Math.Floor(DrawingSurface.ActualWidth * Application.Current.Host.Content.ScaleFactor / 100.0F + 0.5F),
            Math.Floor(DrawingSurface.ActualHeight * Application.Current.Host.Content.ScaleFactor / 100.0F + 0.5F)
            )

        ' Set render resolution to the full native resolution
        D3DInterop.RenderResolution = D3DInterop.NativeResolution

        ' Hook-up native component to DrawingSurface
        DrawingSurface.SetContentProvider(D3DInterop.CreateContentProvider())
        DrawingSurface.SetManipulationHandler(D3DInterop)
    End If
End Sub

private void DrawingSurface_Loaded(object sender, RoutedEventArgs e)
{
    if (m_d3dInterop == null)
    {
        m_d3dInterop = new Direct3DInterop();

        // Set window bounds in dips
        m_d3dInterop.WindowBounds = new Windows.Foundation.Size(
            (float)DrawingSurface.ActualWidth,
            (float)DrawingSurface.ActualHeight
            );

        // Set native resolution in pixels
        m_d3dInterop.NativeResolution = new Windows.Foundation.Size(
            (float)Math.Floor(DrawingSurface.ActualWidth * Application.Current.Host.Content.ScaleFactor / 100.0f + 0.5f),
            (float)Math.Floor(DrawingSurface.ActualHeight * Application.Current.Host.Content.ScaleFactor / 100.0f + 0.5f)
            );

        // Set render resolution to the full native resolution
        m_d3dInterop.RenderResolution = m_d3dInterop.NativeResolution;

        // Hook-up native component to DrawingSurface
        DrawingSurface.SetContentProvider(m_d3dInterop.CreateContentProvider());
        DrawingSurface.SetManipulationHandler(m_d3dInterop);
    }
}

The Windows Runtime component

This section discusses the Windows Runtime component that’s created as part of the XAML and Direct3D app project template. Many of the files in this project handle the low-level details of hooking up the DrawingSurface control with the component. In most scenarios you won’t need to modify them.

Direct3DInterop.h

This header file declares the Direct3DInterop class. This class serves as a proxy between the XAML engine and your Direct3D code. Note that this class implements the IDrawingSurfaceManipulationHandler interface that includes the PointerPressed()()(), PointerMoved()()(), and PointerReleased()()() events.

public ref class Direct3DInterop sealed : public Windows::Phone::Input::Interop::IDrawingSurfaceManipulationHandler

Direct3DInterop.cpp

In the Direct3DInterop implementation file, you’ll see the definition of the CreateContentProvider method that’s called from the DrawingSurface_Loaded event handler in MainPage.xaml.cs. This method initializes a new instance of the Direct3DContentProvider class and casts it as an IDrawingSurfaceContentProvider. If you look at the definition for the IDrawingSurfaceContentProvider, you’ll see that it doesn’t have any members defined. This is because it’s not an interface that you implement using your code. The Direct3DContentProvider class is implemented using the Windows Runtime C++ Template Library (WRL). This method casts the class as a Windows Phone Runtime interface so that it can be accessed by the XAML engine. You shouldn’t have to worry about any of these implementation details as you develop your app. They are explained here solely for informational purposes.

IDrawingSurfaceContentProvider^ Direct3DInterop::CreateContentProvider()
{
    ComPtr<Direct3DContentProvider> provider = Make<Direct3DContentProvider>(this);
    return reinterpret_cast<IDrawingSurfaceContentProvider^>(provider.Get());
}

Next, Direct3DInterop.cpp provides an implementation of the SetManipulationHost(DrawingSurfaceManipulationHost) method that’s called from the DrawingSurface_Loaded event handler in MainPage.xaml.cs so that the Direct3DInterop class can receive touch input events from the DrawingSurface control. The template also provides stubbed-out handlers for the touch input events that you can add your own code to.

void Direct3DInterop::SetManipulationHost(DrawingSurfaceManipulationHost^ manipulationHost)
{
    manipulationHost->PointerPressed +=
        ref new TypedEventHandler<DrawingSurfaceManipulationHost^, PointerEventArgs^>(this, &Direct3DInterop::OnPointerPressed);

    manipulationHost->PointerMoved +=
        ref new TypedEventHandler<DrawingSurfaceManipulationHost^, PointerEventArgs^>(this, &Direct3DInterop::OnPointerMoved);

    manipulationHost->PointerReleased +=
        ref new TypedEventHandler<DrawingSurfaceManipulationHost^, PointerEventArgs^>(this, &Direct3DInterop::OnPointerReleased);
}

void Direct3DInterop::OnPointerPressed(DrawingSurfaceManipulationHost^ sender, PointerEventArgs^ args)
{
    // Insert your code here.
}

void Direct3DInterop::OnPointerMoved(DrawingSurfaceManipulationHost^ sender, PointerEventArgs^ args)
{
    // Insert your code here.
}

void Direct3DInterop::OnPointerReleased(DrawingSurfaceManipulationHost^ sender, PointerEventArgs^ args)
{
    // Insert your code here.
}

Next, the setter for the RenderResolution property is defined. You can see that if the renderer, which will be described later, exists, it is updated with the new resolution and then RecreateSychronizedTexture is called to recreate the synchronized texture with the new size.

void Direct3DInterop::RenderResolution::set(Windows::Foundation::Size renderResolution)
{
    if (renderResolution.Width  != m_renderResolution.Width ||
        renderResolution.Height != m_renderResolution.Height)
    {
        m_renderResolution = renderResolution;

        if (m_renderer)
        {
            m_renderer->UpdateForRenderResolutionChange(m_renderResolution.Width, m_renderResolution.Height);
            RecreateSynchronizedTexture();
        }
    }
}

Next, Direct3DInterop.cpp defines Connect and Disconnect methods. Connect is called after the SetContentProvider(Object) call has completed. In this method, a new CubeRenderer is created and its Initialize method is called. This is when the Direct3D graphics device and the synchronized texture are initially created. During this call also is when CubeRenderer creates device-dependent resources like vertex buffers and shaders. Note that the dimensions that were calculated in MainPage.xaml.cs are provided here so that the synchronized texture is created with the correct size.Disconnect is called when the associated DrawingSurface control leaves the XAML visual tree.

HRESULT Direct3DInterop::Connect(_In_ IDrawingSurfaceRuntimeHostNative* host)
{
    m_renderer = ref new CubeRenderer();
    m_renderer->Initialize();
    m_renderer->UpdateForWindowSizeChange(WindowBounds.Width, WindowBounds.Height);
    m_renderer->UpdateForRenderResolutionChange(m_renderResolution.Width, m_renderResolution.Height);

    // Restart timer after renderer has finished initializing.
    m_timer->Reset();

    return S_OK;
}

void Direct3DInterop::Disconnect()
{
    m_renderer = nullptr;
}

The PrepareResources method is called by the XAML engine for each frame before the texture is rendered. If your app is using the DrawingSurface to draw something that doesn’t change with every frame, such as a UI element or a 3d scene that isn’t currently animating, you may want to allow the synchronized texture to be displayed for multiple frames without rerendering it. To do this, you can perform a check in PrepareResources to determine whether the texture should be rerendered and, if not, set the contentDirty variable to false. When contentDirty is false, GetTexture, which is defined next in the file, will not be called for the current frame.

HRESULT Direct3DInterop::PrepareResources(_In_ const LARGE_INTEGER* presentTargetTime, _Out_ BOOL* contentDirty)
{
    *contentDirty = true;

    return S_OK;
}

There are two overloads of the method GetTexture in the Direct3DInterop class. The first one calls Update and Render on the CubeRenderer object. These methods are where your code will update the state of your game and make the Direct3D calls to draw to the shared texture.

The XAML engine will call PrepareResources, shown above, every time it updates the UI. Calling RequestAdditionalFrame tells the XAML engine to redraw the UI as soon as possible after the current frame is done. The call RequestAdditionalFrame can be made from anywhere in the Windows Phone Runtime component after Connect has been called and before Disconnect is called. To minimize power consumption, you can remove the call to RequestAdditionalFrame from GetTexture and instead call it from elsewhere in your code only when you need to redraw. You could wait several seconds between calls, in the case of drawing static content, or you could call it multiple times per second for animated content. Calling RequestAdditionalFrame from inside GetTexture will continuously request a new frame as soon as possible, resulting in the maximum possible framerate for the drawing surface.

HRESULT Direct3DInterop::GetTexture(_In_ const DrawingSurfaceSizeF* size, _Out_ IDrawingSurfaceSynchronizedTextureNative** synchronizedTexture, _Out_ DrawingSurfaceRectF* textureSubRectangle)
{
    m_timer->Update();
    m_renderer->Update(m_timer->Total, m_timer->Delta);
    m_renderer->Render();

    RequestAdditionalFrame();

    return S_OK;
}

The second overload of GetTexture, that takes no arguments, is called by the XAML engine to get a reference to the synchronized texture that’s shared between the DrawingSurface control and the Windows Phone Runtime component. In most scenarios you won’t need to modify the code in this method.

ID3D11Texture2D* Direct3DInterop::GetTexture()
{
    return m_renderer->GetTexture();
}

Direct3DBase.cpp

Direct3DBase is a class you can find in many Direct3D project templates, including those for Direct3D apps for Windows Phone, as well as for Windows 8. The implementation of this class is slightly different for the XAML and Direct3D app template, but its structure and functionality are very similar. The class exposes an Initialize method that immediately calls CreateDeviceResources. This method calls D3D11CreateDevice to create a Direct3D graphics device. Note that a swap chain is not created for the device. This is because the Windows Phone Runtime draws to a synchronized texture, not directly to the screen. The swap chain for this device is created and maintained by the XAML engine, which draws the synchronized texture to the screen.

As discussed earlier in this walkthrough, Direct3DBase exposes the UpdateForWindowSizeChange and UpdateForRenderResolutionChange methods that are called when the DrawingSurface component size is set. UpdateForRenderResolutionChange sets the resources related to the synchronized texture to null and then calls CreateWindowSizeDependentResources which recreates them. These resources include an ID3DTexture2D, which represents the synchronized texture shared between the XAML engine and the Windows Phone Runtime component, as well as a render target view, a depth stencil texture, and a viewport, which are needed to draw to the texture.

CubeRenderer.cpp

The CubeRenderer class inherits from Direct3Dbase. It provides some example Direct3D code that draws a cube to the shared texture. Again, this file is common to Direct3D templates on different platforms. If you’re just getting started with Direct3D, you may want to spend some time just modifying the code in this file to change what’s drawn to the screen. Eventually, you will most likely replace this class with one that you create specifically for your game, but you should keep the same basic structure. For example, your class should inherit from Direct3DBase.

CubeRenderer.cpp contains a method definition for CreateDeviceResources. It first calls the Direct3DBase method of the same name, which is where the graphics device is created. After that call, the method is able to create the resources that are dependent on the device. The example code in the template creates a vertex buffer and an index buffer that define the geometry of the cube, in addition to a vertex shader and a pixel shader that define how the cube is drawn to the shared texture.

Next, the method CreateWindowSizeDependentResources is defined. This calls the Direct3DBase class method of the same name and then creates a projection matrix based on the size of the DrawingSurface control.

Finally, CubeRenderer.cpp implements the Update and Render methods that are called once for each frame. Update usually is where objects in your game are updated to their new position for each frame. In the example code, the view and model matrices are updated to make the cube spin around on the screen. Render is where the Direct3D calls are made to draw to the shared texture. The code you put in this method will vary widely depending on your game, but it is important to note that before rendering, the render target is set to the render target view that was created in Direct3Dbase. This call means that subsequent drawing calls will draw to the texture that is shared between the XAML engine and the Windows Phone Runtime component, as opposed to the device’s back buffer or some other render target.

    m_d3dContext->OMSetRenderTargets(
        1,
        m_renderTargetView.GetAddressOf(),
        m_depthStencilView.Get()
        );

Direct3DContentProvider.cpp

Direct3DContentProvider.cpp handles the low-level details of hooking up the XAML engine with the Windows Phone Runtime component. This behavior was encapsulated in this class because in most cases you won’t need to modify it to implement your game.

One important thing to note is that in this class’s GetTexture method, you’ll see calls to the BeginDraw and EndDraw methods of the shared texture. These calls define the period in which you can draw to the texture. Because the texture is synchronized between threads, attempting to modify the texture before or after this code block will fail. You can still use the app’s graphics device to draw to other textures that you have created, but not to the synchronized texture.

        hr = m_synchronizedTexture->BeginDraw();
        
        if (SUCCEEDED(hr))
        {
            hr = m_controller->GetTexture(size, synchronizedTexture, textureSubRectangle);
        }

        m_synchronizedTexture->EndDraw();