Benefits of GPU-powered HTML5
At MIX 10 we showed how we’re building on new Windows technologies like Direct2D, DirectWrite and XPS to enable Internet Explorer 9 to render all standards-based web content – text, images, video and SVG – using the power of the GPU.
In this blog post we’ll review the major improvements for web developers and users that come from building on these Windows technologies. For more detailed information on Direct2D technologies, see this excellent PDC2008 talk.
Performance, performance, performance
The benefit of building on Direct2D technologies is that the browser makes the most of the underlying PC hardware that is optimized for rendering rich graphics. This results in faster web applications and a higher quality browsing experience for users and web developers.
As IE9 does more work using the GPU, there is less CPU load, enabling other browser subsystems to do more, as well as enabling higher frame rates for smooth animation and video playback.
The GPU is a much better choice for some graphical operations – for example, the GPU executes alpha blending and bilinear image scaling much faster than the CPU, and uses pixel shaders to perform complex per pixel calculations.
Super-fast zoomed browsing
IE9 uses the GPU to scale images and other content, making zoomed browsing very fast – this is what makes the map zooming demo on ietestdrive.com so fast.
Windows is still the only broadly used operating system that allows the users to change the size of all UI elements on the screen to improve readability and legibility on new high DPI displays on laptops and desktop computers. IE9 builds on the work done in Internet Explorer 8 (the first browser to zoom Web page content by default) to ensure that users can easily read the Web on high DPI displays.
Hardware accelerated HTML5 video using Windows Media Foundation
IE9 makes the most of your graphics hardware by using the Windows Media Foundation system to play HTML5 video, using the CPU or the hardware video decoder if available.
The reduction in CPU usage on machines with hardware video decoders greatly improves battery life – for example, in our MIX demo we played two 720p HD videos, using barely 30% CPU on a $400 netbook. (versus 100% CPU usage in other browsers, playing only one HD video while dropping frames.)
The IE video engine decodes video directly with and into the GPU. Once the video frames are decoded, they can be treated like any other bitmap in the graphics pipeline with full compositing and integration into the rendering system.
High quality image and color support
IE9 uses the Windows Imaging Component (WIC) to decode PNG, JPEG and (new for IE9) TIFF and JPEG XR images. For a lot of uses, JPEG XR offers a good compression improvement over JPEG, allowing you to serve higher quality images at the same file size.
In addition to being up to 30% faster than IE8 decoders, the new WIC decoders understand embedded color profiles in images, making IE9 a color-managed browser that understands ICC v2 and v4 profiles.
Text quality and performance
IE9 uses the GPU (via DirectWrite) to do text output – up to twice as fast as IE8, and with higher quality. Text can be smoothly animated in IE9, and sub-pixel positioning is a more faithful representation of the Web (and font) designer’s intent.
(We’ve also heard your feedback about some fonts showing up ‘blurry’ on some systems; we are working on addressing this.)
High quality graphics printing
To do high quality printing of HTML5, you need a high quality print subsystem. Internet Explorer 9 directly converts web content into XPS format when sending output to the printing system.
XPS is a more modern print system with native support for features such as opacity and complex paths, which results in increased fidelity and quality when printing modern web content.
What if my PC doesn’t have a GPU?
Because IE9 is built on Direct2D, it has full software fallback support for every feature. The goal is to improve graphics performance and fidelity even for the small number of machines in the world without GPUs, via high quality software emulation.
New HTML5 experiences – with the power of the GPU
Taken together, all of these GPU-powered capabilities in IE9 make it easier than ever before to create amazing new class of Web applications using same markup. For example, it took just an hour to create the final MIX demo – a rotating carousel of four translucent videos (at the 28:00 mark in the video). The page uses simple HTML and JavaScript - the same markup that runs in other browsers, but with higher quality and performance made possible by harnessing the power of the GPU.
More to come
We are working with our GPU hardware partners to ensure that every Windows user, using any hardware, has a dramatically improved experience when browsing the Web. We want to enable web developers to create a new class of GPU-powered rich web applications that go far beyond what is possible with today’s browsers.
As always, we welcome your feedback.
Thank you,
Frank Olivier
Program Manager
Comments
Anonymous
January 01, 2003
The comment has been removedAnonymous
January 01, 2003
The comment has been removedAnonymous
January 01, 2003
The comment has been removedAnonymous
January 01, 2003
Awesome, IE9 is the new IE6, with all the blings and no standards! :DAnonymous
April 09, 2010
Sure, just read the comments on the youtube video of the Nvidia ION + IE9 http://www.youtube.com/watch?v=oIOee9u7kHs&feature=player_embedded You all like to play with the crowd don't you? Making tests with rigged settings just to make IE9 look cool isn't really nice.Anonymous
April 09, 2010
Yes, the comments at Youtube are as brilliant as most of the comments here. For those that missed this the first time, the YouTube video was made by Nvidia, not MS.Anonymous
April 09, 2010
So given that IE9 makes use of the Windows Media Foundation framework, I take this to mean that IE9 will playback HTML5 video with whatever codec is installed on the system. Will IE9 also bundle Theora and Vorbis to make open video and audio support simpler for the end user?Anonymous
April 09, 2010
The comment has been removedAnonymous
April 09, 2010
will canvas be GPU accellerated as well?Anonymous
April 09, 2010
The comment has been removedAnonymous
April 09, 2010
Does using the Windows Media Foundation for playback mean the browser supports any codec installed on the system in <video> tags? Or does it just support certain codecs?Anonymous
April 09, 2010
This is not GPU-powered HTML5. It's GPU-powered video.Anonymous
April 09, 2010
"ryan: Incorrect. You should watch that video too. They support the MP4 container with h264 video only. For audio, AAC and MP3. They don't support ogg, vorbis, theora, quicktime, real, WMV, WMA, etc." I watched the video. During the question and answer session there was no emphatic ruling out of Theora support beyond an "I don't believe so", and prior to that there was some considerable confusion from the program manager about which container formats were in fact supported. The question of whether IE9's use of the Windows Media Foundation means any installed codec can be used in conjunction with HTML5 video or audio in IE9 was not resolved by the video.Anonymous
April 09, 2010
See here: http://blogs.msdn.com/ie/archive/2010/03/16/html5-hardware-accelerated-first-ie9-platform-preview-available-for-developers.aspx#9979730Anonymous
April 09, 2010
Again and sorry to be repetitive: I consider these eye-glazing features very amazing. They are wonderful things and i'm proud of IE team efforts in this regard ( visual rendering perfomance ). But i would like you ( IE team ) , with my respect, to share with us some info regarding SVG and ACID3 implementation. You were ~28% and ~55% in each respectively the 16th March. How are you doing now? What are your plans? 80% ACID3 on 3rd quarter of 2010? 70% of SVG 1rd quarter of 2011? another dates maybe? Thank you very muchAnonymous
April 09, 2010
What do you think the acid3 test actually means? Most of the real web devs say very little, and the respected columnists do too (e.g. http://blogs.zdnet.com/Bott/?p=1896&tag=col1;post-1896 -- "controversial acid3"). There are other browsers with higher acid3 scores that can't do borders right :(Anonymous
April 09, 2010
@Paul, the software renderer is a part of D2D, not IE9 specifically. And since there is no D2D on XP, there is no software renderer either.Anonymous
April 09, 2010
The comment has been removedAnonymous
April 09, 2010
The comment has been removedAnonymous
April 09, 2010
The comment has been removedAnonymous
April 09, 2010
Is that means using DirectX Shaders in CSS? Cool! I think implementing <canvas> on a DirectX texture is not difficult. Or not <canvas> but...er... <sence> or directly draw graphics on <img>? (for the patent problem.) And here is my UI design for (9) :LOL: http://phenacenn.deviantart.com/art/IE9-User-Interface-Redesigned-160232929Anonymous
April 10, 2010
Would be great if Internet Explorer 9 could display 32 bits bitmaps. Safari already can...Anonymous
April 10, 2010
please support XP ! XP has directx too, and it should have new directx versions as vista and windows 7 . why do you neglect XP users, when there so many of us using this stable OS?Anonymous
April 10, 2010
@someone: maybe because it is a 9 years old OS, obviously technically inferior to Vista/7 on almost any point of view? Seriously, let's just try to move on here. There cannot be backward compatibility forever, this just hurts innovation.Anonymous
April 10, 2010
This sounds great! I can't wait to get the final IE9. Keep up the good work :-)Anonymous
April 10, 2010
I may have found a possible bug: on Windows 7 32 bit, when Microsoft Security Essential's real-time protection is enabled, an orphaned iexplore.exe process keeps running in the background after closing every IE8 window for a variable amount of time (usually about a minute). This process is small in size (usually 3-7 MB) Instead, every iexplore.exe process is closed immediately (as it should be) if Security Essential's real-time protection is disabled. Could someone pass this info on to someone within Microsoft, please? Thank you.Anonymous
April 10, 2010
Please Add Geolocation. Geolocation: http://www.w3.org/TR/geolocation-API/ Geolocation support: Google Chrome 5.0.371.0 FireFox 3.5 Test: http://html5demos.com/geoAnonymous
April 11, 2010
I have an ION based system (my media center) and a none ION Intel Atom netbook. I can confirm that IE9 runs a lot faster with ION. Great work, not only does IE9 catch up, it sets a new standard. As enterprise system administrator, I know IE is the only enterprise able webbrowser. (try packaging the latest official release of firefox or chrome using App-V or ThinApp..., or lockdown via GPO's/GPP, and patching via WSUS) Keep up the good work, and hopefully release an IE9 version that includes a user interface soon. p.s. One tip, please do not include default favorites in IE9, it's quite difficult to remove the default ones in IE8 for new users. Roaming favorites (e.g. via Mesh) would also be very appreciated for home usersAnonymous
April 11, 2010
@ryan Theora and Vorbis are dead. Get used to it..Anonymous
April 11, 2010
@ao: Post flamebait elsewhere, please. Theora and Vorbis are not "dead", they simply won't be supported for the HTML5 video tag in major browsers like Safari and IE. There's little doubt that these formats will remain in use in other scenarios.Anonymous
April 11, 2010
The comment has been removedAnonymous
April 11, 2010
DailyMotion has over 300,000 Theora encoded videos. I'd suggest that's slightly more than 15 minutes worth: http://openvideo.dailymotion.com/ Wikipedia's multimedia is only available in open video and audio formats. Their collection of video has grown significantly of late: http://www.videoonwikipedia.org/ http://videos.videoonwikipedia.org/ Google cares. They've recently funded development of Theora for ARM devices: http://google-opensource.blogspot.com/2010/04/interesting-times-for-video-on-web.html At this point there's no compelling reason not to support Theora and Vorbis in IE9 and I very much hope Microsoft will do so.Anonymous
April 11, 2010
The comment has been removedAnonymous
April 11, 2010
The comment has been removedAnonymous
April 11, 2010
The comment has been removedAnonymous
April 11, 2010
@Piddlemeyer: h.264 costs 5 million bucks (and I think it's PER YEAR) for a content distributor license. I'm not sure Wikipedia has this sort of cash on hand. How long would it take for the 20% less efficient Theora encoded content to reach that extra cost in bandwidth costs? Reading the release note for TheorARM 0.0.3 (the Theora fork intended for ARM tuning pending reintegration upstream), it is already capable of fully decoding 320x240 video at more than full speed on a 400MHz ARMv4 device with no video acceleration whatsoever (55% of the CPU load is spent on YUV to RGB conversion on un-postprocessed video). Your average mobile phone includes a 320x240 screen. If it can playback video, chances are that it includes a hardware YUV to RGB converter. The iPhone has a 480x320 screen, with YUV to RGB and downsampling done in hardware, for example. Meaning that TheorARM could playback full screen video on an iPhone.Anonymous
April 11, 2010
@John Piddlemeyer: «There is no DRM in IE9» Actually, there is but at different places (e.g. embedded fonts support).Anonymous
April 11, 2010
"Don't like Microsoft's decision? Use Firefox and don't bother the rest of us." Heck, I use Minefield. Rather than being angry and negative, why not help build a better Internet: http://a52.video2.blip.tv/7320003802153/Etherworks-WebMadeMovies3BuildingABetterInternet799.oggAnonymous
April 11, 2010
@Mitch74 Actually the license fees for any AVC/h.264 internet video, that is distributed free for end users, are zero at least until 2015. So Wikipedia can host h.264 video without any charge at least untill 2015 (and likely longer)Anonymous
April 11, 2010
@Wurst Sure, EOT fonts have some 'DRM' - type designers (just like most rational human beings) don't want hundreds of hours of their work stolen. EOT DRM is hardly a burden on the average web browser user. What other parts of IE contain DRM?Anonymous
April 12, 2010
The comment has been removedAnonymous
April 12, 2010
thenonhacker: The problem with the "html5test.com" site is that it tests things which are NOT specified in HTML5 (e.g. specific codecs). So it's not a valid test, and shouldn't be trusted until fixed.Anonymous
April 12, 2010
@hAl: The license that's royalty free until 2015 seems to be regarding website hosted content. Reading some of the materials on the MPEG-LA website, it's not clear if this actually permits a royalty-free decoder until 2015. Can you provide citations showing that the royalty exemption applies to browser H.264 decoders?Anonymous
April 12, 2010
@thenonhacker, @Matt: various browsers were tested against the HTML5 test: http://geektechnica.com/2010/04/chrome-passes-html5-test-with-flying-colors/ All browsers passed at least 100 out of 160. Well except for IE8 that passed a pathetic 19. The good news is that IE9 Platform Preview with its GPU enhanced rendering passed a whopping 19 out of 160 tests -- oh wait that's an improvement of zero percent on an originally horrible score. Thanks for trying out for the Future of the Web team! - don't call us, we'll call you.Anonymous
April 12, 2010
Nick: Thanks for sharing this boring FUD... the problem is that this "TEST" doesn't actually test what's in HTML5. And "testing" HTML5 is pretty stupid, at this point, since the spec still changes daily.Anonymous
April 12, 2010
@Nick, as mentioned during the keynote, IE9 Preview #1 does not support the Video or Audio tags; they're coming in a future preview build. Others on the team are more expert in the exact demands of HTML5, but as far as I know, the spec requires no particular codec to be supported, suggesting that testing for particular codecs leads to a misleading score.Anonymous
April 12, 2010
Hello! I would kindly request that a little thing that's been annoying me about the scroll bar be fixed. I use to browse the interwebs with my IE8 window maximized, and often, in a reflexive manner, i tend to swing my pointer all the way to the right to grab hold of the scroll bar and scroll. The problem is that the scroll bar itself ends just short of the rightmost part of IE's window, and so I have to move the pointer a couple of pixels to the left before I can start scrolling. It's not the whole world, but I sure would appreciate it if this was fixed. Oh and good work on IE9 so far! thumbs up Best wishes!Anonymous
April 12, 2010
"I would like to second John Doe's request of shifting the scroll bar slightly to the right when IE maximized. It's just a little thing, but sometimes even the little things can have an impact overall with making web browsing easier, and I feel this would be a positive change. Looking forward to IE9!"Anonymous
April 12, 2010
@hAL: and in 2015, Wikipedia will have to re-encode all their videos, because I betcha the MPEG won't set such a golden goose slide. For a precedent, see Compuserve and GIF compression (that caused the development of the PNG format). Also, Fraunhofer IIS/Thomson and the MPEG 1 layer 3 audio compression algorithm (that caused the development of the Vorbis codec - along with mp3's low quality).Anonymous
April 12, 2010
The comment has been removedAnonymous
April 12, 2010
The comment has been removedAnonymous
April 12, 2010
@Mitch 74: Remember, Theora is based on VP3.2 (and is capable of playing VP3-encoded content). VP3.2 was released in late 2000, so any submarine patents for VP3.2 would actually expire in 2020, and that only applies to algorithms that weren't on On2's previous codecs. Only patents that involve the changes to VP3 added by Theora would last beyond 2020. So, as time goes on, Theora becomes a harder target to hit, patentwise.Anonymous
April 12, 2010
I've noticed a new test on the Preview demo site: ICCv2 and ICCv4. I find it confusing:
- current IE9 preview fails the ICC test: the top and bottom of the mesa don't have the same alpha levels. If I look at the reference image (or enable ICC support in Firefox 3.6), the top and bottom should have identical levels. Will that be fixed in the next preview?
- the sea slug photo pages are structured differently for IE 9 preview and other browsers: I can't find for the life of me the actual reference image. Will ICC profiles be enabled only in IE 9 mode, or will they enabled in all modes (5,7,8,9)? It's been disabled in Firefox because many websites display images with profiles while these are not intended to apply, so I guess the former, but can you confirm?
Anonymous
April 12, 2010
@Mitch74 There can be no patents on any format, there is just the possibility that there are patents on the implementation of a decoder/encoder. (BTW: before the patent on the lossless compression algorithm used in the GIF image format expired there existed ways of writing decoders that did use the patented technique)Anonymous
April 12, 2010
Oops - it seems only Firefox 3.0 had (tentative) ICCv4 support; it's been reduced to ICCv2 in 3.5/3.6 due to an incomplete ICCv4 support. Please consider the reference browser used above to be 'Adobe Safari' (it seems Adobe knows how to implement Adobe colour profiles... Heh).Anonymous
April 13, 2010
Do you guys have any plans to support VP8 for HTML5 video? It's being open-sourced. http://newteevee.com/2010/04/12/google-to-open-source-vp8-for-html5-video/Anonymous
April 13, 2010
The comment has been removedAnonymous
April 13, 2010
@Fez, @JohnDoe: FWIW, as I outlined here: http://www.enhanceie.com/ie/tweaks.asp There's an (unsupported) tweak for making the scrollbar flush right when full-screen. Simply run http://www.enhanceie.com/dl/no3dborder.regAnonymous
April 13, 2010
@EricLaw Thanks Eric! I applied no3dborder.reg and it works like a charm.Anonymous
April 13, 2010
@Mitch74: "libungif" can read any GIFs. Concerning the scrollbar issue, apparently the IE team wasn't caring so much about Fitts's law… ;-)Anonymous
April 13, 2010
Tangentially related to the question about adding support for additional image types before: Will IE9 finally prevent Quicktime from usurping control of standalone PNG images? I have Quicktime set to not handle .PNGs, and for all that the MIME Types button on the current Quicktime control panel is still broken and opens the wrong thing, for some reason opening the Quicktime control panel through the Quicktime Player instead gives me a version with a working MIME Types button and I have it set to not handle .PNGs in there as well. Yet when I open a PNG link in a new browser window, I have to wait around for Quicktime to load up and mess up the image handling. If I disable the quicktime addon in IE then try opening the link it causes the page to come up as a broken image requiring quicktime addon to display!Anonymous
April 14, 2010
@mitch 74 "the top and bottom of the mesa don't have the same alpha levels." Thanks for the bug report; we're not seeing this, but we'll investigate. The actual sea slug (Spanish Shawl) photos are: http://ie.microsoft.com/testdrive/Graphics/49ColorProfiles/c/adobeRGB.jpg http://ie.microsoft.com/testdrive/Graphics/49ColorProfiles/c/Prophoto.jpg http://ie.microsoft.com/testdrive/Graphics/49ColorProfiles/c/sRGB.jpg ICC profiles will be enabled in all modes in IE9Anonymous
April 14, 2010
Hi, Re: Suggestion Your test page http://ie.microsoft.com/testdrive/Graphics/48ImageSupport/ Should include statistics for file size and load times for each of the graphic formats. Idealy each graphic format should be displayed in its own div container instead of by scripted replacement. Loading the page in other browsers I was initially confused by the differences in rendering speed until I realized that each image is rendered as the background of overlayed individual divs. Regards.
