Play audio and video with MediaPlayer

This article shows you how to play media in your Universal Windows app using the MediaPlayer class. With Windows 10, version 1607, significant improvements were made to the media playback APIs, including a simplified single-process design for background audio, automatic integration with the System Media Transport Controls (SMTC), the ability to synchronize multiple media players, the ability to render video frames to a Windows.UI.Composition surface, and an easy interface for creating and scheduling media breaks in your content. To take advantage of these improvements, the recommended best practice for playing media is to use the MediaPlayer class instead of MediaElement for media playback. The lightweight XAML control, MediaPlayerElement, has been introduced to allow you render media content in a XAML page. Many of the playback control and status APIs provided by MediaElement are now available through the new MediaPlaybackSession object. MediaElement continues to function to support backwards compatibility, but no additional features will be added to this class.

This article will walk you through the MediaPlayer features that a typical media playback app will use. Note that MediaPlayer uses the MediaSource class as a container for all media items. This class allows you to load and play media from many different sources, including local files, memory streams, and network sources, all using the same interface. There are also higher-level classes that work with MediaSource, such as MediaPlaybackItem and MediaPlaybackList, that provide more advanced features like playlists and the ability to manage media sources with multiple audio, video, and metadata tracks. For more information on MediaSource and related APIs, see Media items, playlists, and tracks.

Play a media file with MediaPlayer

Basic media playback with MediaPlayer is very simple to implement. First, create a new instance of the MediaPlayer class. Your app can have multiple MediaPlayer instances active at once. Next, set the Source property of the player to an object that implements the IMediaPlaybackSource, such as a MediaSource, a MediaPlaybackItem, or a MediaPlaybackList. In this example, a MediaSource is created from a file in the app's local storage, and then a MediaPlaybackItem is created from the source and then assigned to the player's Source property.

Unlike MediaElement, MediaPlayer does not automatically begin playback by default. You can start playback by calling Play, by setting the AutoPlay property to true, or waiting for the user to initiate playback with the built-in media controls.

mediaPlayer = new MediaPlayer();
mediaPlayer.Source = MediaSource.CreateFromUri(new Uri("ms-appx:///Assets/example_video.mkv"));
mediaPlayer.Play();

When your app is done using a MediaPlayer, you should call the Close method (projected to Dispose in C#) to clean up the resources used by the player.

mediaPlayer.Dispose();

Use MediaPlayerElement to render video in XAML

You can play media in a MediaPlayer without displaying it in XAML, but many media playback apps will want to render the media in a XAML page. To do this, use the lightweight MediaPlayerElement control. Like MediaElement, MediaPlayerElement allows you to specify whether the built-in transport controls should be shown.

<MediaPlayerElement x:Name="_mediaPlayerElement" AreTransportControlsEnabled="False" HorizontalAlignment="Stretch"  Grid.Row="0"/>

You can set the MediaPlayer instance that the element is bound to by calling SetMediaPlayer.

_mediaPlayerElement.SetMediaPlayer(mediaPlayer);

You can also set the playback source on the MediaPlayerElement and the element will automatically create a new MediaPlayer instance that you can access using the MediaPlayer property.

_mediaPlayerElement.Source = MediaSource.CreateFromUri(new Uri("ms-appx:///Assets/example_video.mkv"));
mediaPlayer = _mediaPlayerElement.MediaPlayer;
mediaPlayer.Play();

The MediaPlayer is detached from MediaPlayerElement when the MediaPlayerElement is destroyed or when a new MediaPlayer is set using SetMediaPlayer. When detached, MediaPlayerElement treats the underlying MediaPlayer differently depending on whether it was created by MediaPlayerElement or set using SetMediaPlayer.

If the MediaPlayer was created by MediaPlayerElement, it will properly Close the MediaPlayer for you. If the MediaPlayer was set on MediaPlayerElement using SetMediaPlayer, you are responsible for ensuring the MediaPlayer is properly closed. Failing to do so may result in fatal playback errors in MediaPlayer. The following code snippet shows how to properly detach and close in code.

// Get a reference to the current media source.
IMediaPlaybackSource _mediaPlayerElement = _mediaPlayerElement.Source;

// Pause playback if able.
if (mediaPlayer.PlaybackSession.CanPause)
{
    mediaPlayer.Pause();
}

// Disconnect the MediaPlayer from its source. This can be done by setting 
// the MediaPlayerElement Source property to null or by directly setting the
// source to null on the underlying MediaPlayer.
_mediaPlayerElement.Source = null;

// Disconnect the MediaPlayer from MediaPlayerElement.
_mediaPlayerElement.SetMediaPlayer(null);

// Dispose of the MediaPlayer or Source if they're no longer needed.
if (source is MediaSource mediaSource)
{
    mediaSource.Dispose();
}

mediaPlayer.Dispose();

Common MediaPlayer tasks

This section shows you how to use some of the features of the MediaPlayer.

Set the audio category

Set the AudioCategory property of a MediaPlayer to one of the values of the MediaPlayerAudioCategory enumeration to let the system know what kind of media you are playing. Games should categorize their music streams as GameMedia so that game music mutes automatically if another application plays music in the background. Music or video applications should categorize their streams as Media or Movie so they will take priority over GameMedia streams.

mediaPlayer.AudioCategory = MediaPlayerAudioCategory.Media;

Output to a specific audio endpoint

By default, the audio output from a MediaPlayer is routed to the default audio endpoint for the system, but you can specify a specific audio endpoint that the MediaPlayer should use for output. In the example below, MediaDevice.GetAudioRenderSelector returns a string that uniquely identifies the audio render category of devices. Next, the DeviceInformation method FindAllAsync is called to get a list of all available devices of the selected type. You may programmatically determine which device you want to use or add the returned devices to a ComboBox to allow the user to select a device.

string audioSelector = MediaDevice.GetAudioRenderSelector();
var outputDevices = await DeviceInformation.FindAllAsync(audioSelector);
foreach (var device in outputDevices)
{
    var deviceItem = new ComboBoxItem();
    deviceItem.Content = device.Name;
    deviceItem.Tag = device;
    _audioDeviceComboBox.Items.Add(deviceItem);
}

In the SelectionChanged event for the devices combo box, the AudioDevice property of the MediaPlayer is set to the selected device, which was stored in the Tag property of the ComboBoxItem.

private void _audioDeviceComboBox_SelectionChanged(object sender, SelectionChangedEventArgs e)
{
    DeviceInformation selectedDevice = (DeviceInformation)((ComboBoxItem)_audioDeviceComboBox.SelectedItem).Tag;
    if (selectedDevice != null)
    {
        mediaPlayer.AudioDevice = selectedDevice;
    }
}

Playback session

As described previously in this article, many of the functions that are exposed by the MediaElement class have been moved to the MediaPlaybackSession class. This includes information about the playback state of the player, such as the current playback position, whether the player is paused or playing, and the current playback speed. MediaPlaybackSession also provides several events to notify you when the state changes, including the current buffering and download status of content being played and the natural size and aspect ratio of the currently playing video content.

The following example shows you how to implement a button click handler that skips 10 seconds forward in the content. First, the MediaPlaybackSession object for the player is retrieved with the PlaybackSession property. Next the Position property is set to the current playback position plus 10 seconds.

private void _skipForwardButton_Click(object sender, RoutedEventArgs e)
{
    var session = mediaPlayer.PlaybackSession;
    session.Position = session.Position + TimeSpan.FromSeconds(10);
}

The next example illustrates using a toggle button to toggle between normal playback speed and 2X speed by setting the PlaybackRate property of the session.

private void _speedToggleButton_Checked(object sender, RoutedEventArgs e)
{
    mediaPlayer.PlaybackSession.PlaybackRate = 2.0;
}
private void _speedToggleButton_Unchecked(object sender, RoutedEventArgs e)
{
    mediaPlayer.PlaybackSession.PlaybackRate = 1.0;
}

Starting with Windows 10, version 1803, you can set the rotation with which video is presented in the MediaPlayer in increments of 90 degrees.

mediaPlayer.PlaybackSession.PlaybackRotation = MediaRotation.Clockwise90Degrees;

Detect expected and unexpected buffering

The MediaPlaybackSession object described in the previous section provides two events for detecting when the currently playing media file begins and ends buffering, BufferingStarted and BufferingEnded. This allows you to update your UI to show the user that buffering is occurring. Initial buffering is expected when a media file is first opened or when the user switches to a new item in a playlist. Unexpected buffering can occur when the network speed degrades or if the content management system providing the content experiences technical issues. Starting with RS3, you can use the BufferingStarted event to determine if the buffering event is expected or if it is unexpected and interrupting playback. You can use this information as telemetry data for your app or media delivery service.

Register handlers for the BufferingStarted and BufferingEnded events to receive buffering state notifications.

mediaPlayer.PlaybackSession.BufferingStarted += MediaPlaybackSession_BufferingStarted;
mediaPlayer.PlaybackSession.BufferingEnded += MediaPlaybackSession_BufferingEnded;

In the BufferingStarted event handler, cast the event args passed into the event to a MediaPlaybackSessionBufferingStartedEventArgs object and check the IsPlaybackInterruption property. If this value is true, the buffering that triggered the event is unexpected and interrupting playback. Otherwise, it is expected initial buffering.

private void MediaPlaybackSession_BufferingStarted(MediaPlaybackSession sender, object args)
{
    MediaPlaybackSessionBufferingStartedEventArgs bufferingStartedEventArgs = args as MediaPlaybackSessionBufferingStartedEventArgs;
    if (bufferingStartedEventArgs != null && bufferingStartedEventArgs.IsPlaybackInterruption)
    {
        // update the playback quality telemetry report to indicate that
        // playback was interrupted
    }

    // update the UI to indicate that playback is buffering
}
private void MediaPlaybackSession_BufferingEnded(MediaPlaybackSession sender, object args)
{
    // update the UI to indicate that playback is no longer buffering
}

Pinch and zoom video

MediaPlayer allows you to specify the source rectangle within video content that should be rendered, effectively allowing you to zoom into video. The rectangle you specify is relative to a normalized rectangle (0,0,1,1) where 0,0 is the upper left hand of the frame and 1,1 specifies the full width and height of the frame. So, for example, to set the zoom rectangle so that the top-right quadrant of the video is rendered, you would specify the rectangle (.5,0,.5,.5). It is important that you check your values to make sure that your source rectangle is within the (0,0,1,1) normalized rectangle. Attempting to set a value outside of this range will cause an exception to be thrown.

To implement pinch and zoom using multi-touch gestures, you must first specify which gestures you want to support. In this example, scale and translate gestures are requested. The ManipulationDelta event is raised when one of the subscribed gestures occurs. The DoubleTapped event will be used to reset the zoom to the full frame.

_mediaPlayerElement.ManipulationMode = ManipulationModes.Scale | ManipulationModes.TranslateX | ManipulationModes.TranslateY;
_mediaPlayerElement.ManipulationDelta += _mediaPlayerElement_ManipulationDelta;
_mediaPlayerElement.DoubleTapped += _mediaPlayerElement_DoubleTapped;

Next, declare a Rect object that will store the current zoom source rectangle.

Rect _sourceRect = new Rect(0, 0, 1, 1);

The ManipulationDelta handler adjusts either the scale or the translation of the zoom rectangle. If the delta scale value is not 1, it means that the user performed a pinch gesture. If the value is greater than 1, the source rectangle should be made smaller to zoom into the content. If the value is less than 1, then the source rectangle should be made bigger to zoom out. Before setting the new scale values, the resulting rectangle is checked to make sure it lies entirely within the (0,0,1,1) limits.

If the scale value is 1, then the translation gesture is handled. The rectangle is simply translated by the number of pixels in gesture divided by the width and height of the control. Again, the resulting rectangle is checked to make sure it lies within the (0,0,1,1) bounds.

Finally, the NormalizedSourceRect of the MediaPlaybackSession is set to the newly adjusted rectangle, specifying the area within the video frame that should be rendered.

private void _mediaPlayerElement_ManipulationDelta(object sender, ManipulationDeltaRoutedEventArgs e)
{

    if (e.Delta.Scale != 1)
    {
        var halfWidth = _sourceRect.Width / 2;
        var halfHeight = _sourceRect.Height / 2;

        var centerX = _sourceRect.X + halfWidth;
        var centerY = _sourceRect.Y + halfHeight;

        var scale = e.Delta.Scale;
        var newHalfWidth = (_sourceRect.Width * e.Delta.Scale) / 2;
        var newHalfHeight = (_sourceRect.Height * e.Delta.Scale) / 2;

        if (centerX - newHalfWidth > 0 && centerX + newHalfWidth <= 1.0 &&
            centerY - newHalfHeight > 0 && centerY + newHalfHeight <= 1.0)
        {
            _sourceRect.X = centerX - newHalfWidth;
            _sourceRect.Y = centerY - newHalfHeight;
            _sourceRect.Width *= e.Delta.Scale;
            _sourceRect.Height *= e.Delta.Scale;
        }
    }
    else
    {
        var translateX = -1 * e.Delta.Translation.X / _mediaPlayerElement.ActualWidth;
        var translateY = -1 * e.Delta.Translation.Y / _mediaPlayerElement.ActualHeight;

        if (_sourceRect.X + translateX >= 0 && _sourceRect.X + _sourceRect.Width + translateX <= 1.0 &&
            _sourceRect.Y + translateY >= 0 && _sourceRect.Y + _sourceRect.Height + translateY <= 1.0)
        {
            _sourceRect.X += translateX;
            _sourceRect.Y += translateY;
        }
    }

    mediaPlayer.PlaybackSession.NormalizedSourceRect = _sourceRect;
}

In the DoubleTapped event handler, the source rectangle is set back to (0,0,1,1) to cause the entire video frame to be rendered.

private void _mediaPlayerElement_DoubleTapped(object sender, DoubleTappedRoutedEventArgs e)
{
    _sourceRect = new Rect(0, 0, 1, 1);
    mediaPlayer.PlaybackSession.NormalizedSourceRect = _sourceRect;
}

NOTE This section describes touch input. Touchpad sends pointer events and will not send Manipulation events.

Handling policy-based playback degradation

In some circumstances the system may degrade the playback of a media item, such as reducing the resolution (constriction), based on a policy rather than a performance issue. For example, video may be degraded by the system if it is being played using an unsigned video driver. You can call MediaPlaybackSession.GetOutputDegradationPolicyState to determine if and why this policy-based degradation is occurring and alert the user or record the reason for telemetry purposes.

The following example shows an implementation of a handler for the MediaPlayer.MediaOpened event that is raised when the player opens a new media item. GetOutputDegradationPolicyState is called on the MediaPlayer passed into the handler. The value of VideoConstrictionReason indicates the policy reason that the video is constricted. If the value isn't None, this example logs the degradation reason for telemetry purposes. This example also shows setting the bitrate of the AdaptiveMediaSource currently being played to the lowest bandwidth to save data usage, since the video is constricted and won't be displayed at high resolution anyway. For more information on using AdaptiveMediaSource, see Adaptive streaming.

private void MediaPlayer_MediaOpened(MediaPlayer sender, object args)
{
    MediaPlaybackSessionOutputDegradationPolicyState info = sender.PlaybackSession.GetOutputDegradationPolicyState();

    if (info.VideoConstrictionReason != MediaPlaybackSessionVideoConstrictionReason.None)
    {
        // Switch to lowest bitrate to save bandwidth
        adaptiveMediaSource.DesiredMaxBitrate = adaptiveMediaSource.AvailableBitrates[0];

        // Log the degradation reason or show a message to the user
        System.Diagnostics.Debug.WriteLine("Logging constriction reason: " + info.VideoConstrictionReason);
    }
}

Use MediaPlayerSurface to render video to a Windows.UI.Composition surface

Starting with Windows 10, version 1607, you can use MediaPlayer to render video to an ICompositionSurface, which allows the player to interoperate with the APIs in the Windows.UI.Composition namespace. The composition framework allows you to work with graphics in the visual layer between XAML and the low-level DirectX graphics APIs. This enables scenarios like rendering video into any XAML control. For more information on using the composition APIs, see Visual Layer.

The following example illustrates how to render video player content onto a Canvas control. The media player-specific calls in this example are SetSurfaceSize and GetSurface. SetSurfaceSize tells the system the size of the buffer that should be allocated for rendering content. GetSurface takes a Compositor as an argument and retrieves an instance of the MediaPlayerSurface class. This class provides access to the MediaPlayer and Compositor used to create the surface and exposes the surface itself through the CompositionSurface property.

The rest of the code in this example creates a SpriteVisual to which the video is rendered and sets the size to the size of the canvas element that will display the visual. Next a CompositionBrush is created from the MediaPlayerSurface and assigned to the Brush property of the visual. Next a ContainerVisual is created and the SpriteVisual is inserted at the top of its visual tree. Finally, SetElementChildVisual is called to assign the container visual to the Canvas.

mediaPlayer.SetSurfaceSize(new Size(_compositionCanvas.ActualWidth, _compositionCanvas.ActualHeight));

var compositor = ElementCompositionPreview.GetElementVisual(this).Compositor;
MediaPlayerSurface surface = mediaPlayer.GetSurface(compositor);

SpriteVisual spriteVisual = compositor.CreateSpriteVisual();
spriteVisual.Size =
    new System.Numerics.Vector2((float)_compositionCanvas.ActualWidth, (float)_compositionCanvas.ActualHeight);

CompositionBrush brush = compositor.CreateSurfaceBrush(surface.CompositionSurface);
spriteVisual.Brush = brush;

ContainerVisual container = compositor.CreateContainerVisual();
container.Children.InsertAtTop(spriteVisual);

ElementCompositionPreview.SetElementChildVisual(_compositionCanvas, container);

Use MediaTimelineController to synchronize content across multiple players.

As discussed previously in this article, your app can have several MediaPlayer objects active at a time. By default, each MediaPlayer you create operates independently. For some scenarios, such as synchronizing a commentary track to a video, you may want to synchronize the player state, playback position, and playback speed of multiple players. Starting with Windows 10, version 1607, you can implement this behavior by using the MediaTimelineController class.

Implement playback controls

The following example shows how to use a MediaTimelineController to control two instances of MediaPlayer. First, each instance of the MediaPlayer is instantiated and the Source is set to a media file. Next, a new MediaTimelineController is created. For each MediaPlayer, the MediaPlaybackCommandManager associated with each player is disabled by setting the IsEnabled property to false. And then the TimelineController property is set to the timeline controller object.

MediaTimelineController _mediaTimelineController;

mediaPlayer = new MediaPlayer();
mediaPlayer.Source = MediaSource.CreateFromUri(new Uri("ms-appx:///Assets/example_video.mkv"));
_mediaPlayerElement.SetMediaPlayer(mediaPlayer);


_mediaPlayer2 = new MediaPlayer();
_mediaPlayer2.Source = MediaSource.CreateFromUri(new Uri("ms-appx:///Assets/example_video_2.mkv"));
_mediaPlayerElement2.SetMediaPlayer(_mediaPlayer2);

_mediaTimelineController = new MediaTimelineController();

mediaPlayer.CommandManager.IsEnabled = false;
mediaPlayer.TimelineController = _mediaTimelineController;

_mediaPlayer2.CommandManager.IsEnabled = false;
_mediaPlayer2.TimelineController = _mediaTimelineController;

Caution The MediaPlaybackCommandManager provides automatic integration between MediaPlayer and the System Media Transport Controls (SMTC), but this automatic integration can't be used with media players that are controlled with a MediaTimelineController. Therefore you must disable the command manager for the media player before setting the player's timeline controller. Failure to do so will result in an exception being thrown with the following message: "Attaching Media Timeline Controller is blocked because of the current state of the object." For more information on media player integration with the SMTC, see Integrate with the System Media Transport Controls. If you are using a MediaTimelineController you can still control the SMTC manually. For more information, see Manual control of the System Media Transport Controls.

Once you have attached a MediaTimelineController to one or more media players, you can control the playback state by using the methods exposed by the controller. The following example calls Start to begin playback of all associated media players at the beginning of the media.

private void PlayButton_Click(object sender, RoutedEventArgs e)
{
    _mediaTimelineController.Start();
}

This example illustrates pausing and resuming all of the attached media players.

private void PauseButton_Click(object sender, RoutedEventArgs e)
{
    if(_mediaTimelineController.State == MediaTimelineControllerState.Running)
    {
        _mediaTimelineController.Pause();
        _pauseButton.Content = "Resume";
    }
    else
    {
        _mediaTimelineController.Resume();
        _pauseButton.Content = "Pause";
    }
}

To fast-forward all connected media players, set the playback speed to a value greater that 1.

private void FastForwardButton_Click(object sender, RoutedEventArgs e)
{
    _mediaTimelineController.ClockRate = 2.0;
}

The next example shows how to use a Slider control to show the current playback position of the timeline controller relative to the duration of the content of one of the connected media players. First, a new MediaSource is created and a handler for the OpenOperationCompleted of the media source is registered.

var mediaSource = MediaSource.CreateFromUri(new Uri("ms-appx:///Assets/example_video.mkv"));
mediaSource.OpenOperationCompleted += MediaSource_OpenOperationCompleted;
mediaPlayer.Source = mediaSource;
_mediaPlayerElement.SetMediaPlayer(mediaPlayer);

The OpenOperationCompleted handler is used as an opportunity to discover the duration of the media source content. Once the duration is determined, the maximum value of the Slider control is set to the total number of seconds of the media item. The value is set inside a call to RunAsync to make sure it is run on the UI thread.

TimeSpan _duration;

private async void MediaSource_OpenOperationCompleted(MediaSource sender, MediaSourceOpenOperationCompletedEventArgs args)
{
    _duration = sender.Duration.GetValueOrDefault();

    await Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal, () =>
    {
        _positionSlider.Minimum = 0;
        _positionSlider.Maximum = _duration.TotalSeconds;
        _positionSlider.StepFrequency = 1;
    }); 
}

Next, a handler for the timeline controller's PositionChanged event is registered. This is called periodically by the system, approximately 4 times per second.

_mediaTimelineController.PositionChanged += _mediaTimelineController_PositionChanged;

In the handler for PositionChanged, the slider value is updated to reflect the current position of the timeline controller.

private async void _mediaTimelineController_PositionChanged(MediaTimelineController sender, object args)
{
    if (_duration != TimeSpan.Zero)
    {
        await Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal, () =>
        {
            _positionSlider.Value = sender.Position.TotalSeconds / (float)_duration.TotalSeconds;
        });
    }
}

Offset the playback position from the timeline position

In some cases you may want the playback position of one or more media players associated with a timeline controller to be offset from the other players. You can do this by setting the TimelineControllerPositionOffset property of the MediaPlayer object you want to be offset. The following example uses the durations of the content of two media players to set the minimum and maximum values of two slider control to plus and minus the length of the item.

_timelineOffsetSlider1.Minimum = -1 * _duration.TotalSeconds;
_timelineOffsetSlider1.Maximum = _duration.TotalSeconds;
_timelineOffsetSlider1.StepFrequency = 1;

_timelineOffsetSlider2.Minimum = -1 * _duration2.TotalSeconds;
_timelineOffsetSlider2.Maximum = _duration2.TotalSeconds;
_timelineOffsetSlider2.StepFrequency = 1;

In the ValueChanged event for each slider, the TimelineControllerPositionOffset for each player is set to the corresponding value.

private void _timelineOffsetSlider1_ValueChanged(object sender, RangeBaseValueChangedEventArgs e)
{
    mediaPlayer.TimelineControllerPositionOffset = TimeSpan.FromSeconds(_timelineOffsetSlider1.Value);
}

private void _timelineOffsetSlider2_ValueChanged(object sender, RangeBaseValueChangedEventArgs e)
{
    _mediaPlayer2.TimelineControllerPositionOffset = TimeSpan.FromSeconds(_timelineOffsetSlider2.Value);
}

Note that if the offset value of a player maps to a negative playback position, the clip will remain paused until the offset reaches zero and then playback will begin. Likewise, if the offset value maps to a playback position greater than the duration of the media item, the final frame will be shown, just as it does when a single media player reached the end of its content.

Play spherical video with MediaPlayer

Starting with Windows 10, version 1703, MediaPlayer supports equirectangular projection for spherical video playback. Spherical video content is no different from regular, flat video in that MediaPlayer will render the video as long as the video encoding is supported. For spherical video that contains a metadata tag that specifies that the video uses equirectangular projection, MediaPlayer can render the video using a specified field-of-view and view orientation. This enables scenarios such as virtual reality video playback with a head-mounted display or simply allowing the user to pan around within spherical video content using the mouse or keyboard input.

To play back spherical video, use the steps for playing back video content described previously in this article. The one additional step is to register a handler for the MediaPlayer.MediaOpened event. This event gives you an opportunity to enable and control the spherical video playback parameters.

mediaPlayer = new MediaPlayer();
mediaPlayer.MediaOpened += _mediaPlayer_MediaOpened;
mediaPlayer.Source = MediaSource.CreateFromUri(new Uri("ms-appx:///Assets/example_video_spherical.mp4"));
_mediaPlayerElement.SetMediaPlayer(mediaPlayer);
mediaPlayer.Play();

In the MediaOpened handler, first check the frame format of the newly opened media item by checking the PlaybackSession.SphericalVideoProjection.FrameFormat property. If this value is SphericaVideoFrameFormat.Equirectangular, then the system can automatically project the video content. First, set the PlaybackSession.SphericalVideoProjection.IsEnabled property to true. You can also adjust properties such as the view orientation and field of view that the media player will use to project the video content. In this example, the field of view is set to a wide value of 120 degrees by setting the HorizontalFieldOfViewInDegrees property.

If the video content is spherical, but is in a format other than equirectangular, you can implement your own projection algorithm using the media player's frame server mode to receive and process individual frames.

private void _mediaPlayer_MediaOpened(MediaPlayer sender, object args)
{
    if (sender.PlaybackSession.SphericalVideoProjection.FrameFormat == SphericalVideoFrameFormat.Equirectangular)
    {
        sender.PlaybackSession.SphericalVideoProjection.IsEnabled = true;
        sender.PlaybackSession.SphericalVideoProjection.HorizontalFieldOfViewInDegrees = 120;

    }
    else if (sender.PlaybackSession.SphericalVideoProjection.FrameFormat == SphericalVideoFrameFormat.Unsupported)
    {
        // If the spherical format is unsupported, you can use frame server mode to implement a custom projection
    }
}

The following example code illustrates how to adjust the spherical video view orientation using the left and right arrow keys.

protected override void OnKeyDown(KeyRoutedEventArgs e)
{
    if (mediaPlayer.PlaybackSession.SphericalVideoProjection.FrameFormat != SphericalVideoFrameFormat.Equirectangular)
    {
        return;
    }

    switch (e.Key)
    {
        case Windows.System.VirtualKey.Right:
            mediaPlayer.PlaybackSession.SphericalVideoProjection.ViewOrientation *= Quaternion.CreateFromYawPitchRoll(.1f, 0, 0);
            break;
        case Windows.System.VirtualKey.Left:
            mediaPlayer.PlaybackSession.SphericalVideoProjection.ViewOrientation *= Quaternion.CreateFromYawPitchRoll(-.1f, 0, 0);
            break;
    }
}

If your app supports playlists of video, you may want to identify playback items that contain spherical video in your UI. Media playlists are discussed in detail in the article, Media items, playlists, and tracks. The following example shows creating a new playlist, adding an item, and registering a handler for the MediaPlaybackItem.VideoTracksChanged event, which occurs when the video tracks for a media item are resolved.

var playbackList = new MediaPlaybackList();
var item = new MediaPlaybackItem(MediaSource.CreateFromUri(new Uri("ms-appx:///Assets/RIFTCOASTER HD_injected.mp4")));
item.VideoTracksChanged += Item_VideoTracksChanged;
playbackList.Items.Add(item);
mediaPlayer.Source = playbackList;

In the VideoTracksChanged event handler, get the encoding properties for any added video tracks by calling VideoTrack.GetEncodingProperties. If the SphericalVideoFrameFormat property of the encoding properties is a value other than SphericaVideoFrameFormat.None, then the video track contains spherical video and you can update your UI accordingly if you choose.

private void Item_VideoTracksChanged(MediaPlaybackItem sender, IVectorChangedEventArgs args)
{
    if (args.CollectionChange != CollectionChange.ItemInserted)
    {
        return;
    }
    foreach (var videoTrack in sender.VideoTracks)
    {
        if (videoTrack.GetEncodingProperties().SphericalVideoFrameFormat != SphericalVideoFrameFormat.None)
        {
            // Optionally indicate in the UI that this item contains spherical video
        }
    }
}

Use MediaPlayer in frame server mode

Starting with Windows 10, version 1703, you can use MediaPlayer in frame server mode. In this mode, the MediaPlayer does not automatically render frames to an associated MediaPlayerElement. Instead, your app copies the current frame from the MediaPlayer to an object that implements IDirect3DSurface. The primary scenario this feature enables is using pixel shaders to process video frames provided by the MediaPlayer. Your app is responsible for displaying each frame after processing, such as by showing the frame in a XAML Image control.

In the following example, a new MediaPlayer is initialized and video content is loaded. Next, a handler for VideoFrameAvailable is registered. Frame server mode is enabled by setting the MediaPlayer object's IsVideoFrameServerEnabled property to true. Finally, media playback is started with a call to Play.

mediaPlayer = new MediaPlayer();
mediaPlayer.Source = MediaSource.CreateFromUri(new Uri("ms-appx:///Assets/example_video.mkv"));
mediaPlayer.VideoFrameAvailable += mediaPlayer_VideoFrameAvailable;
mediaPlayer.IsVideoFrameServerEnabled = true;
mediaPlayer.Play();

The next example shows a handler for VideoFrameAvailable that uses Win2D to add a simple blur effect to each frame of a video and then displays the processed frames in a XAML Image control.

Whenever the VideoFrameAvailable handler is called, the CopyFrameToVideoSurface method is used to copy the contents of the frame to an IDirect3DSurface. You can also use CopyFrameToStereoscopicVideoSurfaces to copy 3D content into two surfaces, for processing the left eye and right eye content separately. To get an object that implements IDirect3DSurface this example creates a SoftwareBitmap and then uses that object to create a Win2D CanvasBitmap, which implements the necessary interface. A CanvasImageSource is a Win2D object that can be used as the source for an Image control, so a new one is created and set as the source for the Image in which the content will be displayed. Next, a CanvasDrawingSession is created. This is used by Win2D to render the blur effect.

Once all of the necessary objects have been instantiated, CopyFrameToVideoSurface is called, which copies the current frame from the MediaPlayer into the CanvasBitmap. Next, a Win2D GaussianBlurEffect is created, with the CanvasBitmap set as the source of the operation. Finally, CanvasDrawingSession.DrawImage is called to draw the source image, with the blur effect applied, into the CanvasImageSource that has been associated with Image control, causing it to be drawn in the UI.

private async void mediaPlayer_VideoFrameAvailable(MediaPlayer sender, object args)
{
    CanvasDevice canvasDevice = CanvasDevice.GetSharedDevice();

    await Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal, () =>
    {
        if(frameServerDest == null)
        {
            // FrameServerImage in this example is a XAML image control
            frameServerDest = new SoftwareBitmap(BitmapPixelFormat.Rgba8, (int)FrameServerImage.Width, (int)FrameServerImage.Height, BitmapAlphaMode.Ignore);
        }
        if(canvasImageSource == null)
        {
            canvasImageSource = new CanvasImageSource(canvasDevice, (int)FrameServerImage.Width, (int)FrameServerImage.Height, DisplayInformation.GetForCurrentView().LogicalDpi);//96); 
            FrameServerImage.Source = canvasImageSource;
        }

        using (CanvasBitmap inputBitmap = CanvasBitmap.CreateFromSoftwareBitmap(canvasDevice, frameServerDest))
        using (CanvasDrawingSession ds = canvasImageSource.CreateDrawingSession(Windows.UI.Colors.Black))
        {

            mediaPlayer.CopyFrameToVideoSurface(inputBitmap);

            var gaussianBlurEffect = new GaussianBlurEffect
            {
                Source = inputBitmap,
                BlurAmount = 5f,
                Optimization = EffectOptimization.Speed
            };

            ds.DrawImage(gaussianBlurEffect);

        }
    });
}

private void FrameServerSubtitlesButton_Click(object sender, RoutedEventArgs e)
{

    mediaPlayer = new MediaPlayer();
    var source = MediaSource.CreateFromUri(new Uri("ms-appx:///Assets/example_video.mkv"));
    var item = new MediaPlaybackItem(source);

    item.TimedMetadataTracksChanged += Item_TimedMetadataTracksChanged;


    mediaPlayer.Source = item;
    mediaPlayer.VideoFrameAvailable += mediaPlayer_VideoFrameAvailable_Subtitle;
    mediaPlayer.IsVideoFrameServerEnabled = true;
    mediaPlayer.Play();

    mediaPlayer.IsMuted = true;

}

private void Item_TimedMetadataTracksChanged(MediaPlaybackItem sender, IVectorChangedEventArgs args)
{
    if(sender.TimedMetadataTracks.Count > 0)
    {
        sender.TimedMetadataTracks.SetPresentationMode(0, TimedMetadataTrackPresentationMode.PlatformPresented);
    }
}

private async void mediaPlayer_VideoFrameAvailable_Subtitle(MediaPlayer sender, object args)
{
    CanvasDevice canvasDevice = CanvasDevice.GetSharedDevice();

    await Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal, () =>
    {
        if (frameServerDest == null)
        {
            // FrameServerImage in this example is a XAML image control
            frameServerDest = new SoftwareBitmap(BitmapPixelFormat.Rgba8, (int)FrameServerImage.Width, (int)FrameServerImage.Height, BitmapAlphaMode.Ignore);
        }
        if (canvasImageSource == null)
        {
            canvasImageSource = new CanvasImageSource(canvasDevice, (int)FrameServerImage.Width, (int)FrameServerImage.Height, DisplayInformation.GetForCurrentView().LogicalDpi);//96); 
            FrameServerImage.Source = canvasImageSource;
        }

        using (CanvasBitmap inputBitmap = CanvasBitmap.CreateFromSoftwareBitmap(canvasDevice, frameServerDest))
        {
            using (CanvasDrawingSession ds = canvasImageSource.CreateDrawingSession(Windows.UI.Colors.Black))
            {

                mediaPlayer.CopyFrameToVideoSurface(inputBitmap);

                //Rect subtitleTargetRect = new Rect(0, 0, inputBitmap.Bounds.Width, inputBitmap.Bounds.Bottom * .1);
                Rect subtitleTargetRect = new Rect(0, 0, 100, 100);

                mediaPlayer.RenderSubtitlesToSurface(inputBitmap);//, subtitleTargetRect);

                //var gaussianBlurEffect = new GaussianBlurEffect
                //{
                //    Source = inputBitmap,
                //    BlurAmount = 5f,
                //    Optimization = EffectOptimization.Speed
                //};

                //ds.DrawImage(gaussianBlurEffect);

                ds.DrawImage(inputBitmap);
            }
        }
    });
}

For more information on Win2D, see the Win2D GitHub repository. To try out the sample code shown above, you will need to add the Win2D NuGet package to your project with the following instructions.

To add the Win2D NuGet package to your effect project

1. In Solution Explorer, right-click your project and select Manage NuGet Packages.
2. At the top of the window, select the Browse tab.
3. In the search box, enter Win2D.
4. Select Win2D.uwp, and then select Install in the right pane.
5. The Review Changes dialog shows you the package to be installed. Click OK.
6. Accept the package license.

Detect and respond to audio level changes by the system

Starting with Windows 10, version 1803, your app can detect when the system lowers or mutes the audio level of a currently playing MediaPlayer. For example, the system may lower, or "duck", the audio playback level when an alarm is ringing. The system will mute your app when it goes into the background if your app has not declared the backgroundMediaPlayback capability in the app manifest. The AudioStateMonitor class allows you to register to receive an event when the system modifies the volume of an audio stream. Access the AudioStateMonitor property of a MediaPlayer and register a handler for the SoundLevelChanged event to be notified when the audio level for that MediaPlayer is changed by the system.

mediaPlayer.AudioStateMonitor.SoundLevelChanged += AudioStateMonitor_SoundLevelChanged;

When handling the SoundLevelChanged event, you may take different actions depending on the type of content being played. If you are currently playing music, then you may want to let the music continue to play while the volume is ducked. If you are playing a podcast, however, you likely want to pause playback while the audio is ducked so the user doesn't miss any of the content.

This example declares a variable to track whether the currently playing content is a podcast, it is assumed that you set this to the appropriate value when selecting the content for the MediaPlayer. We also create a class variable to track when we pause playback programmatically when the audio level changes.

bool isPodcast;
bool isPausedDueToAudioStateMonitor;

In the SoundLevelChanged event handler, check the SoundLevel property of the AudioStateMonitor sender to determine the new sound level. This example checks to see if the new sound level is full volume, meaning the system has stopped muting or ducking the volume, or if the sound level has been lowered but is playing non-podcast content. If either of these are true and the content was previously paused programmatically, playback is resumed. If the new sound level is muted or if the current content is a podcast and the sound level is low, playback is paused, and the variable is set to track that the pause was initiated programmatically.

private void AudioStateMonitor_SoundLevelChanged(Windows.Media.Audio.AudioStateMonitor sender, object args)
{
    if ((sender.SoundLevel == SoundLevel.Full) || (sender.SoundLevel == SoundLevel.Low && !isPodcast))
    {
        if (isPausedDueToAudioStateMonitor)
        {
            mediaPlayer.Play();
            isPausedDueToAudioStateMonitor = false;
        }
    }
    else if ((sender.SoundLevel == SoundLevel.Muted) ||
         (sender.SoundLevel == SoundLevel.Low && isPodcast))
    {
        if (mediaPlayer.PlaybackSession.PlaybackState == MediaPlaybackState.Playing)
        {
            mediaPlayer.Pause();
            isPausedDueToAudioStateMonitor = true;
        }
    }

}

The user may decide that they want to pause or continue playback, even if the audio is ducked by the system. This example shows event handlers for a play and a pause button. In the pause button click handler is paused, if playback had already been paused programmatically, then we update the variable to indicate that the user has paused the content. In the play button click handler, we resume playback and clear our tracking variable.

private void PauseButton_User_Click(object sender, RoutedEventArgs e)
{
    if (isPausedDueToAudioStateMonitor)
    {
        isPausedDueToAudioStateMonitor = false;
    }
    else
    {
        mediaPlayer.Pause();
    }
}

public void PlayButton_User_Click()
{
    isPausedDueToAudioStateMonitor = false;
    mediaPlayer.Play();
}

Related topics

• Media playback
• Media items, playlists, and tracks
• Integrate with the System Media Transport Controls
• Create, schedule, and manage media breaks
• Play media in the background