Using Smooth Streaming Media Element for Windows Phone 7

Article
12/06/2010

In my previous post I described the avaliability of Smooth Streaming Media Element (SSME) for Windows Phone (WP) as a part of the Integrated Media Platform vision for IIS Media Services. In this post I would show you how you can use SSME for WP7 in order to reproduce Smooth Streaming (SS) content live and ondemand. In this post I'll show you how it is possible use directly SSME in a WP project but I would remember that if you are looking for a complete player solution you can leverage the Silverlight Media Framework that give to you a complete solution based on SSME compatible also with WP .

For first you can download the dll that contain SSME for WP from the latest version of Smooth Streaming Client SDK . After intalled the SS Client SDK you can found the Microsoft.Web.Media.SmoothStreaming.dll for WP7 in (your drive):\Program Files (x86)\Microsoft SDKs\IIS Smooth Streaming Client\ select the Windows Phone version.

In order to start with WP you have to download and Install Visual Studio for Windows Phone Tool and Expression Encoder for Windows Phone . You can read all the info for starting with Windows Phone here.

If you are intrested in a global player that works in every page of your app and in a live smooth streaming audio only, you can read for more info this additional post on this topics.

You can start a new Windows Phone project from Visual Studio using the specific template project:

And after that you can add a reference to Microsoft.Web.Media.SmoothStreaming.dll for WP :

In order to insert the SSME in your project you have to add a reference to your MainPage.xaml page, and insert this reference in the <phone:PhoneApplicationPage tag:

xmlns:SSME="clr-namespace:Microsoft.Web.Media.SmoothStreaming;assembly=Microsoft.Web.Media.SmoothStreaming"

I suggest to change also the Orientation Support on the page in order to have the landscape orientation that is more useful for wotching videos:

SupportedOrientations="Landscape" Orientation="Landscape"

this means that you have this PhoneApplicationPage tag in your mainpage.xaml:

<phone:PhoneApplicationPage
    x:Class="WindowsPhoneSSMESample.MainPage"
    xmlns="https://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="https://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:phone="clr-namespace:Microsoft.Phone.Controls;assembly=Microsoft.Phone"
    xmlns:shell="clr-namespace:Microsoft.Phone.Shell;assembly=Microsoft.Phone"
    xmlns:d="https://schemas.microsoft.com/expression/blend/2008"
    xmlns:mc="https://schemas.openxmlformats.org/markup-compatibility/2006"
    xmlns:SSME="clr-namespace:Microsoft.Web.Media.SmoothStreaming;assembly=Microsoft.Web.Media.SmoothStreaming"
    mc:Ignorable="d" d:DesignWidth="728" d:DesignHeight="480"

    FontFamily="{StaticResource PhoneFontFamilyNormal}"
    FontSize="{StaticResource PhoneFontSizeNormal}"
    Foreground="{StaticResource PhoneForegroundBrush}"
    SupportedOrientations="Landscape" Orientation="Landscape"
    shell:SystemTray.IsVisible="True" Loaded="PhoneApplicationPage_Loaded">

In this way you have referenced a namespace SSME that point to the control that are in Microsoft.Web.Media.SmoothStreaming and you could insert these in your xaml page. Now you can add in ContentPanel Grid this tag in order to insert SSME in your page:

<Grid x:Name="ContentPanel" Grid.Row="1" Margin="12,0,12,0"></Grid>
            <SSME:SmoothStreamingMediaElement x:Name="video"/>
        </Grid>

We can use video as a reference from our code to the instance of SSME inserted in the page. From the Loaded Event in MainPage.xaml we can manage the reference and set the SmoothStreamingSource property to the uri of the manifest SS content that we want reproduce:

video.SmoothStreamingSource = new Uri("https://localhost/adaptivestreaming/video/vc1/big_buck_bunny_1080p_surround.ism/manifest");

You can use Play , Stop, Pause method to control reproduction and you can subscribe all the event in the SSME like video.CurrentStateChanged or video.MediaFailed or video.ManifestReady to manage the lifecycle of the SSME in the same way that is possible do for SSME for PC (https://msdn.microsoft.com/en-us/library/ee958035(v=VS.90).aspx) .In order to access to the SSME from the code you have to add to your .cs file this namespace:

using Microsoft.Web.Media.SmoothStreaming;

For example in order to manage Play, Pause, audio volume etc and to show status of Streaming and current Level of bitrates involved in the reproduction we can add these elements to our xaml:

<Grid x:Name="LayoutRoot" Background="Transparent">

        <Grid.RowDefinitions>

            <RowDefinition Height="0.90*"></RowDefinition>

            <RowDefinition Height="0.10*"></RowDefinition>

        </Grid.RowDefinitions>

        <SSME:SmoothStreamingMediaElement x:Name="video" Grid.Row="0" />

        <StackPanel Orientation="Horizontal" Grid.Row="1">

I removed the content grid and added directly in the first layaout grid in our mainpage.xaml:

- two buttons used for Play and Stop directly attached to an handler for click event in the code described before;

- two textblocks used to present status and current bitrates used in reproduction

On the code in .cs files I added this handler and additional code in the loaded event:

private void PhoneApplicationPage_Loaded(object sender, RoutedEventArgs e)
        {
            video.CurrentStateChanged += new RoutedEventHandler(video_CurrentStateChanged);
            video.PlaybackTrackChanged +=new EventHandler<TrackChangedEventArgs>(video_PlaybackTrackChanged);
            video.SmoothStreamingSource = new Uri("https://localhost/adaptivestreaming/CESARONI_DVD02Video.ism/manifest");

        }

        void video_PlaybackTrackChanged(object sender, TrackChangedEventArgs e)
        {
            currentBitrate.Text = e.NewTrack.Bitrate.ToString();
        }

        void video_CurrentStateChanged(object sender, RoutedEventArgs e)
        {
            status.Text = video.CurrentState.ToString();
        }

        private void StopButton_Click(object sender, RoutedEventArgs e)
        {
            //This should simply stop the playback
            video.Stop();
            //We should also reflect the chang on the play button
            PlayButton.Content = "Play";
        }

        private void PlayButton_Loaded(object sender, RoutedEventArgs e)
        {
            switch (video.AutoPlay)
            {
                case false:
                    PlayButton.Content = "Play";
                    break;
                case true:
                    PlayButton.Content = "Pause";
                    break;
            }
        }

private void PlayButton_Click(object sender, RoutedEventArgs e)
{

            //Monitor the state of the content to determine the right action to take on this button being clicked
            //and then change the text to reflect the next action
            switch (video.CurrentState)
            {
                case SmoothStreamingMediaElementState.Playing:
                    video.Pause();
                    PlayButton.Content = "Play";
                    break;
                case SmoothStreamingMediaElementState.Stopped:
                case SmoothStreamingMediaElementState.Paused:
                    video.Play();
                    PlayButton.Content = "Pause";
                    break;
            }

}

As you can see before I subrcibed in Loaded event two events :

video.CurrentStateChanged += new RoutedEventHandler(video_CurrentStateChanged);
video.PlaybackTrackChanged +=new EventHandler<TrackChangedEventArgs>(video_PlaybackTrackChanged);

The first is used to catch change status in SSME and in the handler I write in a specifc TextBlock the current status of SSME, in the second we capture changes in bitrates level used in reproduction and also in this case in the handler I write the current level of bitrate in a specific TextBlock.

The others handlers added to the code are conected to the click event and are used to manage the play stop pause on the SSME. You could read in the code before how this works.

If you would avoid that the application is deactiveted when the Phone go idle or lock (for example you stream audio content and would that the user continue to listen the content ) , you can add to the Loaded Event this code:

PhoneApplicationService.Current.ApplicationIdleDetectionMode = IdleDetectionMode.Disabled;

that prevent deactivation of the app like you can read here and in the specific api description in sdk of wp .

With SSME for WP7 you can play also audio only streaming.

In order to reproduce SS content with SSME on WP7 you have to pay attention to some additional rules about the type of content supported in the current version of WP7 and SSME 1.5. Here you have a list of supported codecs for SS: VC1 / H.264 for video, WMA Pro/HEAAC/AACL for audio.

The Maximum Resolution admited on the W7 for H.264 are :

720 x 480 pixels at 30 fps
720 x 576 pixels at 25 fps

The Current profile for H264 supported on the phone are baseline and main Level 3.0.

It is nesessary PAY ATTENTION TO THE FACT THAT IN CURRENT VERSION OF WP IN SOME HW MULTIPLE RESOLUTIONS BITRATE ISN'T SUPPORTED this means that it is necessary have the same resolution for all the bitrates inlcuded in the range of levels to reproduce in SSME for WP7. You can check if the device supports multiple resolutions during a play with this api https://msdn.microsoft.com/en-us/library/microsoft.phone.info.mediacapabilities.ismultiresolutionvideosupported(v=VS.92).aspx. In the case that the devices doesnt support multiple resolution and you have a previous encoded content with multiple resolutions or you want create a single asset for multiple devices , you have to pay attention to this and in the case you can use the StreamInfo.RestrictTracks API in a ManifestReady event , in order to restrict the range of bitrates used in reproduction . For example imagine to have a manifest that have for biterates <= of 600K the same resolutions and for bitrates > 400k a variable resolutions in order to meet quality requirements of others devices like a PC, in the case tha the is multipleresolutionvideo supported respond with false you can add the code below in the manifest ready event on WP in order to limit the bitarates used in the reproduction :

void video_ManifestReady(object sender, EventArgs e)
        {
            if (video == null)
            {
                return;
            }

            foreach (SegmentInfo segment in video.ManifestInfo.Segments)
            {
                foreach (StreamInfo streaminfo in segment.AvailableStreams)
                {
                    if (MediaStreamType.Video == streaminfo.Type)
                    {
                        List<TrackInfo> tracks = new List<TrackInfo>();

                        foreach (TrackInfo track in streaminfo.AvailableTracks)
                        {
                            if (track.Bitrate <= 400000)
                            {
                                tracks.Add(track);
                            }
                        }
                        streaminfo.RestrictTracks(tracks);
                    }
                }
            }
        }

You can subcribe the ManifestReady event in SSME called video in this way : video.ManifestReady += new EventHandler<EventArgs>(video_ManifestReady);

In alternative you can create an adhoc manifest, server side , one for every devices and point every devices to the right manifest.

Propose an encoding profile is difficult because there are a lot of parameters to consider like bandwith avaliable type of content etc, but on my experience in others project that leverare SS for WP7 I can suggest to have in the manifest of your content this generic profile:

Use Main Level 3.0 with 640x352 for 16:9 with this levels: 1000 k(Full framerate but <=30) , 800k (Full framerate but <=30) , 600k (Full framerate but <=30) , 400k (Full framerate but <=30) , 200k (half framerate <=15), 100 k(half framerate <=15)

If you would serve the same content to WP7, PC and Apple Devices like iPhone and iPad leveraging the capability of IIS Media Services 4.0 and IIS Transform Manager , you have to built a profile using H264 Baseline Level 3.0 and pay attention to the resolutions in differents bitrates levels and Transmux the SS also in Apple Adaptive Streaming.

If you have to stream a to low level of the maximu 1000k suggested before , you can leverage these profiles for WP7 and iPhone/iPad and in the case add more level fo PC using variable resolution for profiles > 600k and use RestrictTracks API on PC for the minimu bitrates for WP7 for the maximum bitrate and during the transmux operation on IIS Transform Manager and IIS MS 4,0 (in the case of live streaming ) limiting profile fo Apple devices to 600k :

4:3 use Baseline Level 3.0 with 480x360 with this levels: 600 k(Full framerate but <=30) , 400k (Full framerate but <=30) , 200k (half framerate <=15), 100 k(half framerate <=15)

16:9 use Baseline Level 3.0 with 480x272 with this levels: 600 k(Full framerate but <=30) , 400k (Full framerate but <=30) , 200k (half framerate <=15), 100 k(half framerate <=15)

If you don't need to cover old iPhone you could move to Main Level 3.0 with the same resolution in order to increase the quality.

Pay attention to the fact that profiles suggested before are only a sample and not necessary these profile are good for your scenarios. It is necessary test it and adjust it to your requirements in order to have the best solution.

Comments

Anonymous
December 06, 2010
Very good. It's Fantastic.
Anonymous
December 16, 2010
The comment has been removed
Anonymous
December 16, 2010
Is there a way to acheive audio only playback mode using SSME? I'm running in to invalidoperation exception while trying to do that. Here is my sample code Player.SmoothStreamingPlaybackMode = PlaybackMode.AudioOnly; Player.SmoothStreamingSource = null; Player.Source = null; Player.SmoothStreamingSource = new Uri("http://xxxx/song.mp3"); Player.Source = new Uri("http://xxxx/song.mp3"); Player.Play(); Any help would be apprecaited.
Anonymous
December 16, 2010
The comment has been removed
Anonymous
December 17, 2010
Thanks Giuseppe. I already use mediaelement in my app, but have seen issues where the quality of the streaming is not that great. So, i wanted to try adaptive streaming using SSME and see if there is any difference.
Anonymous
December 17, 2010
Vinit, in order to use Smooth Streaming it is necessary encode content in this format . You can't use .mp3 , it is necessary produce a Smooth Streaming content. I suggest to you this post blogs.msdn.com/.../move-forward-internet-video-streaming-iis7-media-service-and-smooth-streaming.aspx and this article to start to learn more about SS learn.iis.net/.../smooth-streaming-primer
Anonymous
December 20, 2010
Hi Giuseppe, Thank you so much for all the information. This is really helpful and i think i'll stick with mediaelement for now. I have another question for you. I want to add few mms radio links to my app. The problem is mediaelement doesnt support mms. Smoothstreaming does work if i hardcode the source in the xaml file. If i try to use binding, it doesnt work. it keeps complaining that the source is not set. How do i fix that? Thanks in advance. -Vinit
Anonymous
December 20, 2010
Hi , on WP7 with Silverlight MediaElement you could ms-wmsp ove http, it works but it isn't supported . In order to put at work Windows Media Services (wms) live streaming on WP7 MediaElement it is necessary create a pubblishing point that stream wms ove http. The file extension url of the stream MUST match the encoded source. For example in the case of audio streaming URL must have .wma extension and protocol prefix MUST have http://. For example you can have : http://servername/pubpointname.wma and you can use from a MediaElement: Player.AutoPlay=false; Player.Source = new Uri("http://servername/pubpointname.wma "); Player.Play(); and streaming works. It is not supported but works. It is important correct extension that MUST match the encoded source and wms ove http. I suggest to you to try to use Smooth Streaming using an harware encoder or Expression Encoder 4 Pro. If you have to stream audio only with Smooth Streaming you could do this usinng a campatible encoder and SSME for WP7. You could use also Expression Encoder 4 Pro but current version doesn't suppurt audio anly streaming but you can use a trick: you can select a single profile video with a very low level of bitrate and insert anytype of video signal in input to EE 4 and insert in input the audio that you want stream. In the client side using SSME for WP7 you can select stream audio only and you can ignore the video signal and consume audio only
Anonymous
January 15, 2011
Can we use this to play a playlist in the server? Can the source be www.myserver.com/list.asx Doesn't seem to work.
Anonymous
February 21, 2011
Giuseppe, Great article which has been very helpful. I'm struggling with a couple of idle related issues. I'm trying to play streamed MMS content using the advice above (this works perfectly), however upon playback the application refuses to dim and lock the screen after the usual time period. I'm guessing that mediaelement playback stops this happening. Can I override this effect? Ideally I'd like to use ApplicationIdleDetectionMode, however even when it is set as part of the loaded event it seems to only partially work. When I press the lock button, the audio stops. When I press it again and de-activate the lock screen, the application appears to launch without "resuming". My ultimate goal is for the user to start playback and for the screen to dim and lock whilst the audio continues to play uninterrupted. Basically mimicking the way the built-in music player works.... Thanks for your thoughts on this!
Anonymous
February 28, 2011
Hi Giuseppe great work but i have a small problem with your example which is that it doesn't play the video unless i put the video.SmoothStreamingSource in the public mainpage () to start the video straight away. can you check my source files please? gazcorb at hotmail dot com cheers
Anonymous
March 02, 2011
ugh, that bunny content makes my xap 600 MB? what the deal?
Anonymous
July 04, 2011
Wonderful post! Is it possible to provide a buffer object as a source in either of MediaElement or SSME? I haven't come across such a scenario. Have you tried it Giuseppe?
Anonymous
September 26, 2011
The comment has been removed
Anonymous
September 26, 2011
Thanks for your post !
Anonymous
August 06, 2013
Hello! How can I play the video (in smooth streaming) in fullscreen?

Share via

Using Smooth Streaming Media Element for Windows Phone 7

Comments

Additional resources