Rendering with XNA Framework 4.0 inside of a WPF application

First some prefaces:

1. This isn’t any official solution to embedding XNA Framework graphics inside a WPF application. This is simply my personal attempt at making it happen.
2. The code here might not be the most optimized, but it certainly gets the job done.

Alright, so yesterday I decided I wanted to start building some tools for games. I had nothing in particular at the time, but knew I wanted to use WPF for the application and XNA Framework for the graphics. Since HiDef is now supported in the XNA Game Studio 4.0 Beta, I decided to go ahead and use Visual Studio 2010 with .NET 4.0 and XNA Framework 4.0. Then the hard part came of how to make WPF and XNA Framework play nice together.

After some searching the interwebs, I found a few ways people had accomplished this:

1. Use the regular WinForm approach and use the WindowsFormsHost inside of WPF. This works but I thought it a bit ugly.
2. Render a frameless window over any panels that were XNA Framework content. This works but also is quite ugly. You’re essentially going to be running an application per view and just hiding the fact that it’s a separate application from the user. Not very nice.
3. Use the D3DImage class to display the content. This seems like the ideal, but since the XNA Framework does not directly expose the native IDirect3D9Surface pointer, the only way to make this work is with a bunch of reflection hackery.

So I decided to do it my own way. My first idea was from looking at the D3DImage approach. I wasn’t going to use the D3DImage, but I did start to think about how to create my own ImageSource such that I could use the WPF Image class to display the content. After heading down the MSDN documentation road for the ImageSource and trying to implement it myself, I quickly found it near impossible to decipher exactly how the Image queried the ImageSource for the pixel data.

I then decided that I wouldn’t even implement my own ImageSource, but rather I’d just stick data into an existing ImageSource, the WriteableBitmap to be precise. So I made a class to wrap up a RenderTarget2D along with a WriteableBitmap and handle transferring data from one to the other:

 /// <summary>
/// A wrapper for a RenderTarget2D and WriteableBitmap 
/// that handles taking the XNA rendering and moving it 
/// into the WriteableBitmap which is consumed as the
/// ImageSource for an Image control.
/// </summary>
public class XnaImageSource : IDisposable
{
    // the render target we draw to
    private RenderTarget2D renderTarget;

    // a WriteableBitmap we copy the pixels into for 
    // display into the Image
    private WriteableBitmap writeableBitmap;

    // a buffer array that gets the data from the render target
    private byte[] buffer;

    /// <summary>
    /// Gets the render target used for this image source.
    /// </summary>
    public RenderTarget2D RenderTarget
    {
        get { return renderTarget; }
    }

    /// <summary>
    /// Gets the underlying WriteableBitmap that can 
    /// be bound as an ImageSource.
    /// </summary>
    public WriteableBitmap WriteableBitmap
    {
        get { return writeableBitmap; }
    }

    /// <summary>
    /// Creates a new XnaImageSource.
    /// </summary>
    /// <param name="graphics">The GraphicsDevice to use.</param>
    /// <param name="width">The width of the image source.</param>
    /// <param name="height">The height of the image source.</param>
    public XnaImageSource(GraphicsDevice graphics, int width, int height)
    {
        // create the render target and buffer to hold the data
        renderTarget = new RenderTarget2D(
            graphics, width, height, false,
            SurfaceFormat.Color,
            DepthFormat.Depth24Stencil8);
        buffer = new byte[width * height * 4];
        writeableBitmap = new WriteableBitmap(
            width, height, 96, 96,
            PixelFormats.Bgra32, null);
    }

    ~XnaImageSource()
    {
        Dispose(false);
    }

    public void Dispose()
    {
        Dispose(true);
    }

    protected virtual void Dispose(bool disposing)
    {
        renderTarget.Dispose();

        if (disposing)
            GC.SuppressFinalize(this);
    }

    /// <summary>
    /// Commits the render target data into our underlying bitmap source.
    /// </summary>
    public void Commit()
    {
        // get the data from the render target
        renderTarget.GetData(buffer);

        // because the only 32 bit pixel format for WPF is 
        // BGRA but XNA is all RGBA, we have to swap the R 
        // and B bytes for each pixel
        for (int i = 0; i < buffer.Length - 2; i += 4)
        {
            byte r = buffer[i];
            buffer[i] = buffer[i + 2];
            buffer[i + 2] = r;
        }

        // write our pixels into the bitmap source
        writeableBitmap.Lock();
        Marshal.Copy(buffer, 0, writeableBitmap.BackBuffer, buffer.Length);
        writeableBitmap.AddDirtyRect(
            new Int32Rect(0, 0, renderTarget.Width, renderTarget.Height));
        writeableBitmap.Unlock();
    }
}

With that in place, I now had a mechanism for rendering XNA Framework content (into the RenderTarget2D) and moving that into the WriteableBitmap which could be consumed by WPF.

Next I set out to make a custom XnaControl that I could add to my window. The XAML for the XnaControl is very minimal given that it just wraps an Image:

 <UserControl x:Class="MyApp.XnaControl"
        xmlns="https://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="https://schemas.microsoft.com/winfx/2006/xaml"
        MinHeight="50" MinWidth="50" Background="CornflowerBlue">
    <Image x:Name="rootImage" />
</UserControl>

Nothing special there. I just set it up with a minimum size of 50x50 because that seemed good to me and of course used a CornflowerBlue background so that I can see the control more easily in design view. Then I just have a basic Image which will get hooked up in code.

Next I wrote up the codebehind for my XnaControl. The primary responsibility of the codebehind is to handle setting up the GraphicsDevice along with our XnaImageSource and managing that during rendering. The first bit I needed was to borrow the GraphicsDeviceService.cs from the WinForms Sample and modify it a bit to my liking. Namely since I am using one RenderTarget2D per view, the backbuffer size doesn’t matter so I can remove some parameters and methods for that. I also had to update it to work with the new APIs in XNA Framework 4.0 and eventually ended up with this:

 /// <summary>
/// Helper class responsible for creating and managing the GraphicsDevice.
/// All GraphicsDeviceControl instances share the same GraphicsDeviceService,
/// so even though there can be many controls, there will only ever be a 
/// single underlying GraphicsDevice. This implements the standard 
/// IGraphicsDeviceService interface, which provides notification events for 
/// when the device is reset or disposed.
/// </summary>
public class GraphicsDeviceService : IGraphicsDeviceService
{
    // Singleton device service instance.
    private static GraphicsDeviceService singletonInstance;

    // Keep track of how many controls are sharing the singletonInstance.
    private static int referenceCount;

    /// <summary>
    /// Gets the single instance of the service class for the application.
    /// </summary>
    public static GraphicsDeviceService Instance
    {
        get
        {
            if (singletonInstance == null)
                singletonInstance = new GraphicsDeviceService();
            return singletonInstance;
        }
    }

    // Store the current device settings.
    private PresentationParameters parameters;

    /// <summary>
    /// Gets the current graphics device.
    /// </summary>
    public GraphicsDevice GraphicsDevice { get; private set; }

    // IGraphicsDeviceService events.
    public event EventHandler<EventArgs> DeviceCreated;
    public event EventHandler<EventArgs> DeviceDisposing;
    public event EventHandler<EventArgs> DeviceReset;
    public event EventHandler<EventArgs> DeviceResetting;

    /// <summary>
    /// Constructor is private, because this is a singleton class:
    /// client controls should use the public AddRef method instead.
    /// </summary>
    GraphicsDeviceService() { }

    /// <summary>
    /// Creates the GraphicsDevice for the service.
    /// </summary>
    private void CreateDevice(IntPtr windowHandle)
    {
        parameters = new PresentationParameters();

        // since we're using render targets anyway, the 
        // backbuffer size is somewhat irrelevant
        parameters.BackBufferWidth = 480;
        parameters.BackBufferHeight = 320;
        parameters.BackBufferFormat = SurfaceFormat.Color;
        parameters.DeviceWindowHandle = windowHandle;
        parameters.DepthStencilFormat = DepthFormat.Depth24Stencil8;
        parameters.IsFullScreen = false;

        GraphicsDevice = new GraphicsDevice(
            GraphicsAdapter.DefaultAdapter, 
            GraphicsProfile.HiDef, 
            parameters);

        if (DeviceCreated != null)
            DeviceCreated(this, EventArgs.Empty);
    }

    /// <summary>
    /// Gets a reference to the singleton instance.
    /// </summary>
    public static GraphicsDeviceService AddRef(IntPtr windowHandle)
    {
        // Increment the "how many controls sharing the device" 
        // reference count.
        if (Interlocked.Increment(ref referenceCount) == 1)
        {
            // If this is the first control to start using the
            // device, we must create the device.
            Instance.CreateDevice(windowHandle);
        }

        return singletonInstance;
    }

    /// <summary>
    /// Releases a reference to the singleton instance.
    /// </summary>
    public void Release()
    {
        // Decrement the "how many controls sharing the device" 
        // reference count.
        if (Interlocked.Decrement(ref referenceCount) == 0)
        {
            // If this is the last control to finish using the
            // device, we should dispose the singleton instance.
            if (DeviceDisposing != null)
                DeviceDisposing(this, EventArgs.Empty);

            GraphicsDevice.Dispose();

            GraphicsDevice = null;
        }
    }
}

If you’ve seen the WinForm version, you’ll see it’s fairly similar with a few things removed here and there. I also moved the device creation to a CreateDevice method instead of the constructor and made a public property to get the instance. This seemed useful to me so external classes can hook the DeviceCreated event (and others) so they can load and manage content.

Now that we’re done there, we can write the XnaControl codebehind which will get us rendering our content into the Image control.

 public partial class XnaControl : UserControl
{
    private GraphicsDeviceService graphicsService;
    private XnaImageSource imageSource;

    /// <summary>
    /// Gets the GraphicsDevice behind the control.
    /// </summary>
    public GraphicsDevice GraphicsDevice
    {
        get { return graphicsService.GraphicsDevice; }
    }

    /// <summary>
    /// Invoked when the XnaControl needs to be redrawn.
    /// </summary>
    public Action<GraphicsDevice> DrawFunction;

    public XnaControl()
    {
        InitializeComponent();

        // hook up an event to fire when the control has finished loading
        Loaded += new RoutedEventHandler(XnaControl_Loaded);
    }

    ~XnaControl()
    {
        imageSource.Dispose();

        // release on finalizer to clean up the graphics device
        if (graphicsService != null)
            graphicsService.Release();
    }

    void XnaControl_Loaded(object sender, RoutedEventArgs e)
    {
        // if we're not in design mode, initialize the graphics device
        if (DesignerProperties.GetIsInDesignMode(this) == false)
        {
            InitializeGraphicsDevice();
        }
    }

    protected override void OnRenderSizeChanged(SizeChangedInfo sizeInfo)
    {
        // if we're not in design mode, recreate the 
        // image source for the new size
        if (DesignerProperties.GetIsInDesignMode(this) == false && 
            graphicsService != null)
        {
            // recreate the image source
            imageSource.Dispose();
            imageSource = new XnaImageSource(
                GraphicsDevice, (int)ActualWidth, (int)ActualHeight);
            rootImage.Source = imageSource.WriteableBitmap;
        }

        base.OnRenderSizeChanged(sizeInfo);
    }

    private void InitializeGraphicsDevice()
    {
        if (graphicsService == null)
        {
            // add a reference to the graphics device
            graphicsService = GraphicsDeviceService.AddRef(
                (PresentationSource.FromVisual(this) as HwndSource).Handle);

            // create the image source
            imageSource = new XnaImageSource(
                GraphicsDevice, (int)ActualWidth, (int)ActualHeight);
            rootImage.Source = imageSource.WriteableBitmap;

            // hook the rendering event
            CompositionTarget.Rendering += CompositionTarget_Rendering;
        }
    }

    /// <summary>
    /// Draws the control and allows subclasses to override 
    /// the default behavior of delegating the rendering.
    /// </summary>
    protected virtual void Render()
    {
        // invoke the draw delegate so someone will draw something pretty
        if (DrawFunction != null)
            DrawFunction(GraphicsDevice);
    }

    void CompositionTarget_Rendering(object sender, EventArgs e)
    {
        // set the image source render target
        GraphicsDevice.SetRenderTarget(imageSource.RenderTarget);

        // allow the control to draw
        Render();

        // unset the render target
        GraphicsDevice.SetRenderTarget(null);

        // commit the changes to the image source
        imageSource.Commit();
    }
}

Again, nothing in here too crazy if you read the comments. Two notes in here about performance:

1. Anytime the control is resized, we recreate the image source. We do this because we at least need to fix up the WriteableBitmap and at this point I wasn’t too worried about the performance cost of disposing and recreating the render target.
2. We draw everytime WPF’s Rendering event fires which is 60fps. This is a bit overkill for most scenarios where you only care about drawing when something has changed. It might be worth investigating a way to allow both scenarios with the use of a flag.

Now that we’re all set up, we can actually use the control in a window. Simply add it to the XAML and hook the DrawFunction delegate. Add code to clear the backbuffer and run it. You should see the view drawn using whatever color you cleared to. You could also hook the DeviceCreated event of the GraphicsDeviceService (in your main window’s constructor) and create a SpriteBatch along with a Texture2D (using the new Texture2D.FromStream method) and draw that.

Having the rendering is one great step forward. However most people will want to load in some content to use. I started again with the WinForms Content Sample and started working on upgrading it to use the correct MSBuild APIs. A lot of what was valid for that sample is now considered deprecated so we need to update some of the code to get things working. The ErrorLogger and ServiceContainer are fine as they are, it’s the ContentBuilder we will draw our attention to.

First we have to update the xnaVersion constant to use the 4.0.0.0 version (same public key token) and I also added the rest of the pipeline assemblies:

 // What importers or processors should we load?
private const string xnaVersion = 
    ", Version=4.0.0.0, PublicKeyToken=6d5c3888ef60e27d";
private static readonly string[] pipelineAssemblies =
{
    "Microsoft.Xna.Framework.Content.Pipeline.AudioImporters" + xnaVersion,
    "Microsoft.Xna.Framework.Content.Pipeline.EffectImporter" + xnaVersion,
    "Microsoft.Xna.Framework.Content.Pipeline.FBXImporter" + xnaVersion,
    "Microsoft.Xna.Framework.Content.Pipeline.TextureImporter" + xnaVersion,
    "Microsoft.Xna.Framework.Content.Pipeline.VideoImporters" + xnaVersion,
    "Microsoft.Xna.Framework.Content.Pipeline.XImporter" + xnaVersion,
};

Next we want to remove all using statements for Microsoft.Build.* because the ones already there aren’t the ones we want to use. Instead, replace them with

 using Microsoft.Build.Evaluation;

Now head down into the fields and find the BuildEngine which we want to replace with a ProjectCollection. Project is still valid but it is now a Project type from a different namespace.

 // MSBuild objects used to dynamically build content.
private ProjectCollection projectCollection;
private Project msBuildProject;

With those changes we now have to fix up our methods one by one to get things building. First the CreateBuildProject method which creates the MSBuild project and fills in the basic information. If you compare the WinForms sample with this code, you’ll see it’s quite similar; there are just a few APIs that needed adjustment to fit everything in.

 /// <summary>
/// Creates a temporary MSBuild content project in memory.
/// </summary>
void CreateBuildProject()
{
    string projectPath = 
        Path.Combine(buildDirectory, "content.contentproj");
    string outputPath = Path.Combine(buildDirectory, "bin");

    // Create the project collection
    projectCollection = new ProjectCollection();

    // Hook up our custom error logger.
    errorLogger = new ErrorLogger();
    projectCollection.RegisterLogger(errorLogger);

    // Create the build project.
    msBuildProject = new Project(projectCollection);
    msBuildProject.FullPath = projectPath;

    // set up the properties we care about
    msBuildProject.SetProperty("XnaPlatform", "Windows");
    msBuildProject.SetProperty("XnaFrameworkVersion", "v4.0");
    msBuildProject.SetProperty("XnaProfile", "HiDef");
    msBuildProject.SetProperty("Configuration", "Release");
    msBuildProject.SetProperty("OutputPath", outputPath);

    // Register any custom importers or processors.
    foreach (string pipelineAssembly in pipelineAssemblies)
    {
        msBuildProject.AddItem("Reference", pipelineAssembly);
    }

    // Include the standard targets file that defines
    // how to build XNA Framework content.
    msBuildProject.Xml.AddImport(
        "$(MSBuildExtensionsPath)\\Microsoft\\XNA Game Studio\\v4.0\\" +
        "Microsoft.Xna.GameStudio.ContentPipeline.targets");
}

Next up is the Add method which adds a piece of content to our project. Again, nothing has really changed here; we’ve just restructured it around the new APIs for the Microsoft.Build.Evaluation.Project type.

 /// <summary>
/// Adds a new content file to the MSBuild project. The importer and
/// processor are optional: if you leave the importer null, it will
/// be autodetected based on the file extension, and if you leave the
/// processor null, data will be passed through without any processing.
/// </summary>
public void Add(
    string filename, string name, 
    string importer, string processor)
{
    // set up the metadata for this item
    var metadata = new SortedList<string,string>();
    metadata.Add("Link", Path.GetFileName(filename));
    metadata.Add("Name", name);

    if (!string.IsNullOrEmpty(importer))
        metadata.Add("Importer", importer);

    if (!string.IsNullOrEmpty(processor))
        metadata.Add("Processor", processor);

    // add the item
    msBuildProject.AddItem("Compile", filename, metadata);
}

Lastly we have Clear. Clear used to be a one liner, but the method it used doesn’t exist anymore. Instead we use LINQ to find all of the items and then remove them using a new RemoveItems method.

 /// <summary>
/// Removes all content files from the MSBuild project.
/// </summary>
public void Clear()
{
    // select all compiled objects in the project and remove them
    var compileObjects = from i in project.Items 
                         where i.ItemType == "Compile" 
                         select i;
    project.RemoveItems(compileObjects);
}

Now we’ve updated ContentBuilder to use .NET 4.0 and are ready to  go. From here on it’s exactly like the WinForms Content Sample in how you create the ContentBuilder, ContentManager, and so on, so I’ll just let you look it up there instead of pasting it all here.

And there we go. XNA Framework 4.0 rendering inside of a WPF application. Hopefully this helps you get up and running with your own apps leveraging these great technologies.

Comments

• Anonymous
  July 25, 2010
  Great article!  It's also nice you did this with all public and supported APIs. A few suggestions related to performance: The WriteableBitmap has a nice API, but it surely isn't the fastest because of its double buffering (and a few other CPU bound operations).  You may try out the InteropBitmap as the only time the CPU touches the pixels is to upload it to the GPU.  It is more complicated to use, but I have an example here in one of my projects:  silverlightviewport.codeplex.com/.../39341 The color conversion code will have a performance hit due to .NET bounds checking on arrays.  You can get around this by using pointers in C# and the "fixed" keyword.  An alternative would be to use a WPF pixel shader to do the conversion on the GPU. Copying from the GPU to system memory is generally much slower than system memory to GPU.  There's not much you can do about it in this case.  I'm not too familiar with XNA, but in D3D, copying from a GPU surface -> System Memory surface -> Get pixels was hugely faster than GPU surface -> Get pixels. Great job on this and thanks for the article and code! -Jer

• Anonymous
  July 25, 2010
  Yeah great read Nick!  Very interesting topic and I'm sure this will come in handy.

• Anonymous
  July 28, 2010
  Nick, I work for a Software Company based in Dublin. I would be really interested in having a chat with you in relation to your experience and previous projects that you have worked on. Please email me @ nikki.mitchell@vero.ie I look forward to hearing from you.

• Anonymous
  July 29, 2010
  Sorry, but where can i find code for this? Thanks

• Anonymous
  July 30, 2010
  @Amer: All the code you should need is in the linked WinForms samples and posted in the blog. I'm not hosting a full sample, sorry.

• Anonymous
  September 02, 2010
  I ran into an issue where implementing this kept crashing VS on me whenever I'd recompile. I finally thought to attach another instance of VS to debug what was causing the crash. Turns out the destructor for XnaControl was running. I ultimately ended up using a Dispatcher.Invoke-based solution (by the time I'd gotten there I'd added many extra calls to DesignerProperties.GetIsInDesignMode - which throws an InvalidOperationException if the thread that created the control isn't the one calling it) though in retrospect a simple check to see if imageSource was null might have solved it as well. The Dispatcher.Invoke solution required taking the logic from the destructor, putting it inside a method "void Destruct() { ... }", adding "delegate void DestructDel();" creating a class field instance of the delegate "DestructDel del;", assigning it to the method in the constructor "del = Destruct;", then calling "Dispatcher.Invoke(del, null);" inside the Destructor instead of calling the code directly. Anyway, thought I'd share in case anyone else was running into a similar issue. Once I solved that, it worked brilliantly!

• Anonymous
  September 07, 2010
  @MikeBMcL: in the code originally posted I experienced that exact problem, so I tried to check whether the image source was null with no success. Your solution works like a charm. Thanks for sharing it!

• Anonymous
  September 26, 2010
  @MikeBMcL I cannot seem to get this to work :( Can you share the exact code you made? This is mine: Class XnaImageSource: private delegate void DestructDel(); private DestructDel del; private void Destruct() {    Dispose(false); } CTOR del = Destruct; DTOR System.Windows.Threading.Dispatcher.CurrentDispatcher.Invoke(del, null);

• Anonymous
  October 19, 2010
  The comment has been removed

• Anonymous
  October 28, 2010
  Check this solution: www.c-sharpcorner.com/.../Default.aspx

• Anonymous
  November 02, 2010
  This is completely counter-productive to the point of this article, but I'm curious if anyone has tried using the IDirect3D9Surface method of WPF integration in XNA 4.  We were using it successfully for XNA 3.1, but RenderTarget2D has changed quite a bit, and I'm not sure where to get the surface pointer any more.

• Anonymous
  November 12, 2010
  @Thraka This is the code I used: public XnaControl() { InitializeComponent(); if (!DesignerProperties.GetIsInDesignMode(this)) { // hook up an event to fire when the control has finished loading Loaded += new RoutedEventHandler(XnaControl_Loaded); } destructDelegate = Destruct; } delegate void DestructDelegate(); void Destruct() { if (DesignerProperties.GetIsInDesignMode(this) == false) { if (imageSource != null) { imageSource.Dispose(); } // release on finalizer to clean up the graphics device if (graphicsDeviceService != null) { graphicsDeviceService.Release(); } } } DestructDelegate destructDelegate; ~XnaControl() { if (imageSource != null || graphicsDeviceService != null) { Dispatcher.Invoke(destructDelegate, null); } }

• Anonymous
  March 16, 2011
  Can any of you guys provide me a download link to a Importer/Processor please? I'm about to take a class in college and we are required to make one, but I simply cannot understand it. Can you please help me, anyone? Just upload it to Rapidshare or Megaupload or something like that please. Thanks or just email the Importer to XeXiiBoss@live.com if possible thanks.

• Anonymous
  April 13, 2011
  Can anybody point me to a complete solution that I can download? I'm apparently missing something; if I try to plug this code all into the Winforms sample posted on the App Hub I still don't see how to load up an Xna game itself into it all. Does anybody have a working solution?

• Anonymous
  May 28, 2011
  To anyone having trouble with this sample, please note: The WinFormsContentLoading has been updated since this article was written.  Following the changes exactly compared to the copy of the source code you download today will get you in a pickle.  Took me a while to figure this out. :)

• Anonymous
  August 24, 2011
  Do you have a download link for a working example? Thanks!

• Anonymous
  September 26, 2011
  Thanks for a great blog and thank you MikeBMcL for solving a very frustrating issue :) Should have checked the comments a bit before..

• Anonymous
  March 22, 2012
  thanks very much. but i don't know how do it.

• Anonymous
  July 26, 2012
  I was able to get rid of the color swap in the commit section of XnaImageSource by preinverting the colors as the last step in the pixel shader, and inverting the clear color before calling clear.  This significantly improves performance at larger screen sizes.

• Anonymous
  March 26, 2014
  Must i write this code in the wpf or xna app? i have both applications and i must embed them urgently, pls someone answer me!

• Anonymous
  April 02, 2014
  I got an error in clear method of ContentBuilder, that part of project.Items, it says project doesn't exist. Anyone has a solution?

• Anonymous
  July 30, 2014
  What kind of project do i must choose to create this project ?