HTML5 Audio and Video Improvements for Windows Phone 8.1

  • Articolo

Internet Explorer on Windows Phone 8.1 adds new media features that greatly expand its support for HTML5 audio and video. Audio and video elements are now fully supported, including inline playback of video content, and adaptive streaming based on the latest Web specifications is supported as well. These new features provide Web site developers the tools they need to provide compelling media experiences, and make IE an ideal browser for mobile media applications.

Support for HTML5 Audio Video Specifications

Internet Explorer in Windows Phone 8.1 now provides full support for HTML5 media. Videos can play natively in the browser without plug-ins, and HTML5 media element events, attributes and properties are fully supported as well. Multiple audio elements can play simultaneously on a single page, making it possible to use HTML5 audio with games or interactive Web apps. And video playback works on a 512MB device. This is a mobile browser, after all!

One newly supported attribute is the Controls Attribute defined in the HTML5 specification. By using it, developers may elect to use the standard IE media playback controls rather than create custom ones.

 <video src="demo.mp4" controls autoplay>
    HTML5 Video is required for this example
</video>

Selecting Default Media Transport Controls Using the HTML5 Controls Attribute

Inline or Full screen playback

By default, videos in IE on Windows Phone 8.1 will play inline in the Web page. The standard playback controls include a full screen button, so that users can choose to play full screen whenever they want.

By default, videos in IE on Windows Phone 8.1 will play inline in the webpage. The standard playback controls include a full screen button, so that users can choose to play full screen whenever they want.

Inline Video Playback with Standard Media Controls including Full Screen Button

Inline video is a great fit for Web pages where video is accompanied by other content (article text, user comments, etc…) that users may want to view while the video is playing. Or sites might want to use a video element to play an animation within the context of other content. These scenarios are easily possible on Windows Phone 8.1 using inline video playback coupled with the HTML5 Video attributes such as controls, mute, autoplay and loop.

Some site designers might prefer to have video playback go directly to full screen. With IE on Windows Phone 8.1, they can use the FullSceen API to provide a true full screen-only experience. For example, Vimeo chose this approach in implementing their support for IE’s HTML5 features. They use the Full Screen API to go directly into full screen video playback when their custom “Play” button is pressed.

Vimeo inline video

User taps on custom Play button

Video plays in full screen by default

Video plays in full screen by default

Media Source Extensions and MPEG-DASH

With the release of Windows 8.1 in 2013, Internet Explorer 11 expanded HTML5 media features to provide plug-in free audio and video streaming that achieved what we called Professional Quality Video – Web video that is just as great for personal videos as it is for premium TV and movies. Internet Explorer 11 achieved this support with a combination of new Web APIs (Media Source Extensions and Encrypted Media Extensions) and standardized formats (MPEG-DASH).

Now Internet Explorer on Windows Phone 8.1 supports Media Source Extensions as well. With it, sites will be able to deliver adaptive streaming videos using the same MPEG-DASH framework to Windows Phone 8.1 devices.

You can try adaptive streaming yourself by playing this Big Buck Bunny Video on our Test Drive site using Internet Explorer on Windows Phone 8.1. The video includes a slider control that lets you change video bitrate on the fly and see the effect on the rendered image a few seconds later.

To create your own adaptive streaming solution Web site, consider using the dash.js framework as a starting point.

Closed Captioning

Last, but not least, Internet Explorer on Windows Phone 8.1 also supports Closed Captioning using the W3C TTML Simple Delivery profile. This is the same caption format introduced with Professional Quality Video on Internet Explorer 11. Captioned videos you target at Internet Explorer 11 will now play on IE on Windows Phone 8.1 as well!

Call to Action

These improvements to the HTML5 audio and video platform for Windows Phone 8.1 make it possible to build great audio and video experiences for both the web browser and web apps.

Here are a few best practices to make sure your videos work correctly on Windows Phone 8.1:

  • Avoid using an <img> overlaid on the video element to provide still preview of the upcoming video. This can work if video is played using an external app, but results in video playing behind the image with inline playback. Instead, use the HTML5 poster attribute to show an image before the video is loaded.
  • Remember to use the controls attribute to use the default media transport controls if you are not building your own. Without these, your users won’t be able to control video playback.
  • Make use of the FullSceen API to give users a full screen option with custom controls, or to launch your videos directly to full screen.
  • Consider implementing MPEG-DASH for adaptive streaming using dash.js as a starting point.

As always, please send feedback to @IEDevChat or file a bug on Connect.

— Jerry D. Smith, Senior Program Manager, Internet Explorer

— Dhruv Chadha, Program Manager, Internet Explorer

Comments

  • Anonymous
    September 26, 2014
    So still no (real) support for Ogg and WebM? The whole non-sense of "if the user has the codec installed" is completely unacceptable. Supporting WebGL but intentionally not supporting free and open source codecs is wildly hypocritical.

  • Anonymous
    September 26, 2014
    @Kathy WebM supports vp8 and vp9, both formats owned and controlled by Google. "Yeah, but Google has made vp free to implement, and it's an open specification." It doesn't matter much as long as these standards are owned and controlled by a private company, which has a goal of making as much money as possible from the assets they own - open specification or not. Think about it for a minute, and ask yourself why Google hasn't let IETF or a similar standards org take ownership of the standards? Why is Google holding on to them? In that perspective it makes WebM nothing more than a spearhead for the expansion of Googled owned formats - privately owned and controlled formats.  

  • Anonymous
    September 26, 2014
    Web doesn't need WebM. h264/5 is far superior in every way.

  • Anonymous
    September 26, 2014
    Note that it's possible to implement alternative codecs in JavaScript; my experimental ogv.js <github.com/.../ogv.js> plays Ogg Theora/Vorbis/Opus files on Safari and desktop IE, and I'm working on integrating it into Wikipedia (where our community has been very conservative about patent-encumbered codecs). However you lose the benefits of having low-level A/V done for you, and even with today's JavaScript engines it's less CPU-efficient than highly tuned SIMD code or GPU hardware. (Hopefully more of the work can move to the GPU with further WebGL and related techs.) My main ask for the IE team is to get Web Audio working -- this will let us do client-side decoding of video with audio on Windows Phone (albeit with resolution limitations on many phone CPUs!) My secondary ask is to work with the other browser makers to devise an actual JavaScript plugin point for media codecs that can provide consistent playback on multiple platforms, and work within the <audio> and <video> elements. This may be trickier... but I think it'd be a big win for future expansion.

  • Anonymous
    September 27, 2014
    Thanks! What about live streaming? Have you guys able to fix this: stackoverflow.com/.../14552820 ? ASF is MSFT's own container, and yet Windows Phone is the only mobile OS which doesn't support MPEG live stream (directly). It was Jan 2013 when I tested with Android and iOS SDK as well as their pre-installed browsers and IE for WP8 was complaining "Can't open this file" (SDK was also rejecting). Whatever happened to people at MSFT who built ASF, AVI, WMV and so many other media formats! Please first support everything which other browsers are supporting: ASF, OGV etc.

  • Anonymous
    September 29, 2014
    www.webmproject.org/.../additional

  • Anonymous
    September 30, 2014
    But what happened to background audio streaming in IE on WP8.1? I could leave a website on WP8, and the music would still be playing. Not anymore. Please, somebody, reply. Taking away features isn't cool.

  • Anonymous
    October 02, 2014
    @Brion Vibber Microsoft agrees with you about Web Audio.  See the reference on https://status.modern.ie/.

  • Anonymous
    October 02, 2014
    @seimis This was a difficult design choice made to accommodate phones with reduced memory.  IE can play music behind a lock screen, but not if another app is opened over it.  Apps can continue to play in background.

  • Anonymous
    March 05, 2015
    The comment has been removed