Measuring Browser Performance with the Windows Performance Tools

We’ve recently discussed the performance characteristics of the Flying Images and A GPU-Powered HTML5 Flickr Photo Viewer samples across different browsers. On the Internet Explorer team we use the Windows Performance Tools to measure browser runtime performance. The Windows Performance Tools are among the most accurate performance tools available which is why they’re commonly used across the industry. In this post I will give an introduction to using the Windows Performance Tools with Internet Explorer.

Using the Windows Performance Tools you can measure not only the overall elapsed time of operations, but also look at the time spent in individual browser and operating system components. In this post we’ll show you how to use the Windows Performance Tools to record and analyze the performance of different browsers. We’ll look at how to understand Windows events, measure CPU and GPU activity, identify working set patterns, and view network activity. We wouldn’t recommend the Windows Performance Tools for the average user; however these tools are a great resource for developers and this post will help you get started using the toolset to understand browser performance.

Background

Microsoft Windows Vista and Windows 7 have an extremely low overhead infrastructure built into the operating system called Event Tracing for Windows which collects performance and system data. Event Tracing for Windows, or ETW for short, enables the Windows operating system and applications to efficiently generate events at runtime. There are hundreds of events that can be captured during execution and later analyzed by tools.

The Windows Performance Tools, or WPT for short, allow you to capture, visualize and analyze ETW traces for your operating system. WPT is part of the Windows 7 SDK and can be downloaded from here. During the installation you can scope the installation to just the “Windows Performance Toolkit”:

First Profile

Once you have the WPT tools installed, you’re ready to start analyzing browser performance.

Let’s started by measuring the performance of Internet Explorer 8 navigating to https://www.bing.com

1. Launch Internet Explorer and navigate to https://www.microsoft.com.

2. Start an elevated command prompt

3. From that command prompt execute

   xperf -start mytrace -on PerfTrack

4. Navigate IE to https://www.bing.com and wait for five seconds after the page appears to be visually done loading and the browser reaches a quiescent state.

5. Stop the trace by executing

   xperf -stop mytrace -d mytrace.etl

6. Now launch the Windows Performance Analyzer, part of the WPT toolkit, by executing

   xperfview mytrace.etl

The last command launches the Windows Performance Analyzer and opens the mytracve.etl file which contains the ETW events that were recorded during the trace.

The Windows Performance Analyzer displays the captured events. In this example you captured events from several providers. Clicking on the ‘ProviderIds’ drop down reveals the 4 providers:

• PerfTrackMetadata
• WinSATAssessment
• Microsoft-PerfTrack-IEFRAME
• Microsoft-PerfTrack-MSHTML

We will go into more detail about providers later in this post.

Notice the time scale at the bottom showing time in seconds. Next we will zoom into the cluster of events. There are two ways to zoom in: a) Ctrl + Scroll-wheel on the mouse, or b) Click + Drag to select a timespan followed by Right-Click on the selection and select ‘Zoom To Selection’.

Next, we will hover over an event to reveal its tooltip listing details about the event. For example, hovering over the first yellow dot shows its tooltip:

The EventName indicates that this event is the Navigation - Start event, which is the beginning of the navigation to bing.com. 

The tooltip of the second yellow dot reveals it as the Navigation - Redirect event. The redirect occurs because we navigated to https://bing.com and the server responds with a redirect to https://www.bing.com. As you know, it is common for sites to redirect users to the primary URL of the home page.

The third yellow dot is the Navigation - Stop event, indicating the navigation to www.bing.com is finished.

Next, we will measure the duration of the redirected navigation to get a sense of the overall time taken.

Click + Drag from the second yellow dot to the third. While dragging, the tooltip shows the duration between the two points in time:

NOTE: The elapsed time between the start and stop events is not meant for comparison among different browsers. Different browsers can name and define events as they see fit. Comparing elapsed times makes sense for a single browser. Attempting to compare them across browsers (or even different versions of the same browser) doesn’t make sense. For example, the ETW events exposed by another browser might have similar names like Navigation Start and Navigation End but they measure very different aspects of the generic term “Navigation”.

However, the elapsed time is interesting for comparing results from multiple traces of the same scenario using the same browser. For example, it is interesting to compare the navigation duration of bing.com and another site. Since the pages are very different the elapsed time will be different. Later, we will look deeper into understanding where the time is spent.

So far, we have looked at the "timeline" view displaying all events captures during the trace. We can drill into events by zooming in and looking at the tooltips of each event. Windows Performance Analyzer has more views that we will look at later.

Providers

We previously mentioned Providers that are the source of the events captured by the Windows Performance Tool Kit. In the previous example, we told xperf to capture events from the "PerfTrack" provider by specifying “PerfTrack” on the command line

xperf -start mytrace -on PerfTrack

We can use Windows Performance Tool Kit to capture and analyze every aspect of the Windows operating system. Each Windows component exposes numerous ETW events so that overall Windows has thousands of events. In addition each application can expose many more, easily leading to information overload.

Depending on the scenario that we are profiling, we chose events from a set of providers so that we focus our attention to the relevant events.

We get a listing of providers, by executing the following command:

xperf -providers

The full list has numerous providers, including Kernel Flags and Kernel Groups at the very end of the list. The following are of interest when looking at Internet Explorer:

• Microsoft-IE
• Microsoft-IEFRAME
• Microsoft-PerfTrack-IEFRAME
• Microsoft-PerfTrack-MSHTML

As well as the Kernel Group:

• Latency

We can use several user mode providers at once, for example:

xperf -start mytrace -on Microsoft-IE+Microsoft-IEFRAME+Microsoft-PerfTrack-IEFRAME+Microsoft-PerfTrack-MSHTML

To use user mode and kernel mode providers at the same time take a look at Example 2 half way down the page.

Tracing Scripts

When we analyze the performance of any component, we run performance trace many times over. To facilitate and make it easier to capture the same traces we use simple batch script. For the examples in this post we used trace.cmd:

 @echo off
set session=mytrace
if not @%1@ == @@ set session=%1

xperf -start %session% -on PerfTrack
if not errorlevel 0 goto :eof

echo.
echo Performance Trace started. 
echo.
echo When done with profile actions, 

pause 

echo.
xperf -stop %session% -d %session%.etl
if not errorlevel 0 goto :eof

echo.
start xperfview %session%.etl

The basic idea of the script is:

• Launch the script from an elevated command prompt
• Reproduce the scenario that we want to capture when the script is waiting for us
• Hit any key in the command prompt to let the script finish the capture and display the result

The following is a slightly more complex script tracek.cmd that also captures kernel mode events:

 @echo off
set session=mytrace
if not @%1@ == @@ set session=%1

xperf -on Latency -f %session%kernel.etl -start %session% -on Microsoft-IE+Microsoft-IEFRAME+Microsoft-PerfTrack-IEFRAME+Microsoft-PerfTrack-MSHTML -f %session%user.etl
 
if not errorlevel 0 goto :eof
 
echo.
echo Performance Trace started. 
echo.
echo When done with profile actions, 
 
pause 
 
echo.
xperf -stop %session%
if not errorlevel 0 goto :eof
xperf -stop
if not errorlevel 0 goto :eof
 
xperf -merge  %session%user.etl %session%kernel.etl %session%combined.etl
if not errorlevel 0 goto :eof
 
echo.
start xperfview %session%combined.etl

Machine Wide Perspective

When we take traces with the second “more complex” script Windows Performance Analyzer shows a holistic view of the entire system during the capture including CPU usage, CPU usage by process, Disk I/O and utilization, process lifetimes, and hard faults along with the Generic Events we looked at before. Each of the additional one can be just as important as CPU usage in determining performance.

We can select which graphs to display by clicking the black shape on the left edge which reveals the Frame List:

We can also overlay multiple graphs to get a combined view. For example, Right+ Click on a graph and select another graph under "Overlay Graph".

Summary Table

Now we will look at a very different view in Windows Performance Analyzer, the “Summary Table” view. Each graph has a “Summary Table” view that is tailored to data in the graph. For example, view the “Summary Table” view of the “Generic Events” graph via Right + Click on the “Generic Events” graph and select “Summary Table”:

We can drill down into the events by provider and reveal the individual event counts and time that they occurred.

Stack Walk

Stack walks provide another dimension when investigating performance profiles, since they let you drill down based on the "Weight", meaning approximate CPU consumed. Since the profile data is based on sampling, it is always a statistical approximation of CPU consumed.

To enable stalk walks, we need to turn them on for the events we are interested in. In this case we are interested in the profile event which is part of the Latency group. For details on enabling stack walking on the event see Stack Walking in XPerf.

A Wide World to Explore

On the IE team we use the Windows Performance Toolkit along with internal tools daily to measure and understand the runtime performance of Internet Explorer. We welcome you to download the tools and start looking at browser performance using the Windows Performance Toolkit on your own systems.

Walter vonKoch

IE Performance Program Manager

Comments

• Anonymous
  June 21, 2010
  You lost credibility as soon as your "navigate to bing" test states you need to start by going to "microsoft.com" - please stop pushing all the MSFT properties when totally unnecessary. I can ping bing.com without pinging any other site first. On an unrelated note, has .innerHTML been fixed in IE9 to work on all elements? (both setting and getting)?

• Anonymous
  June 21, 2010
  Is there ETW provider that will show BHO-related events?

• Anonymous
  June 21, 2010
  @Lost credibility: They tell you to set it to some other site first so that the recored data is only for loading the page, not for loading the browser it's self. And why not choose microsoft.com, last I check this a MICROSOFT blog.

• Anonymous
  June 21, 2010
  @Bill Gates II: While I don't see a use of microsoft.com as a big deal, example.com does exist for a reason. It's certainly less bandwidth-intensive.

• Anonymous
  June 21, 2010
  Since Google has already chosen to branch the poor performing VP8 just a month to create a better performing codec and thus any implementation at the release of IE9 will likely not be compatible with future VP8 video will you reconsider supporting WebM with VP8 in IE9?

• Anonymous
  June 21, 2010
  Why do 'Windows Performance Analyzer' windows have fatter-than-normal (by 2px) border padding? I am genuinely curious. Cf. www.windows7taskforce.com/.../9

• Anonymous
  June 21, 2010
  Posting about performance increases is pointless, IE is slow! and I mean really slow. When IE opens as quickly as Chrome, then post about performance increases.

• Anonymous
  June 21, 2010
  Robert: IE loads as fast or faster than Chrome. Disable your slow browser add-ons and disable automatic proxy detection.

• Anonymous
  June 21, 2010
  Please abridge blog posts with a "Read more" link on the main page. This is annoying.

• Anonymous
  June 22, 2010
  @Matt Still slower. Still less secure. Still less standards-compliant. Any other tips? @Bill Gates II Why should we need to "navigate" to microsoft.com? If the goal was to avoid the cost of loading the browser, why not simply load the browser. The first step should have been "Launch Internet Explorer", NOT "Launch Internet Explorer and navigate to http://www.microsoft.com, then navigate to http://msn.com/, and then navigate to http://msnbc.com/, and then navigate to http://windows.com/, and then navigate to http://live.com/...."

• Anonymous
  June 22, 2010
  The comment has been removed

• Anonymous
  June 22, 2010
  @Ken, IE8 on Windows 7/Vista is the most secure browser. It has Protected Mode support and SmartScreen Filter. Firefox is much more vulnerable than IE8. Or are you still using XP like a Luddite?

• Anonymous
  June 22, 2010
  The comment has been removed

• Anonymous
  June 22, 2010
  @Hand Extensions for other borwser are not part of those browser and allthough an extension like Noscript adds a big layer of security, 90% or more of Firefox users do not use it. A security layer like Smartscreen provided much broader security.  

• Anonymous
  June 22, 2010
  @Matt I do not have any browser add-ons - But why should I have to disable loads of stuff to make IE work like it should to start with. It is still slower, less secure and less standards-compliant. Get over it. @Sam We post about these things to stop Microsoft getting lazy with IE development - There is still a long long way to go before they will have anything to be proud of. I would like to see IE regain some level of respect from the web developer community, but first Microsoft and the IE Team have to have some respect for the web developer community and the W3C.

• Anonymous
  June 23, 2010
  Easy way to speed up IE and reduce its working set:

1. Click Tools, Internet Options, Advanced tab
2. Scroll down and uncheck the 'Enable third-party browser extensions' item
3. Click OK Yes Henry, this will disable all the cutesy toolbars that hang like moss on your browser chrome...

• Anonymous
  June 23, 2010
  The comment has been removed

• Anonymous
  June 23, 2010
  I guess no 7 week between preview this time around...

• Anonymous
  June 23, 2010
  @Neil: What is so magical about your system that you think it represents all systems? IE is as fast or faster on all of mine. @Robert: Please take your rants and silliness elsewhere.

• Anonymous
  June 23, 2010
  @hal: www.zdnet.com/.../6636

• Anonymous
  June 23, 2010
  @Matt: What is so magical about your system that you think it represents all systems? Chrome is faster on all of mine.

• Anonymous
  June 23, 2010
  @Matt: What is so magical about your system that you think it represents all systems? Chrome is faster on all of mine. Tested on various systems - XP (yes people still use it, shock horror) and Win 7.  Virtual and physical.  IBM and Dell hardware, plenty memory for the task.

• Anonymous
  July 04, 2010
  Konqueror is way faster than Chrome, duh!   Just kidding ^.^ This is what I was looking for as tools like IE Dev tools, Firebug, YSlow, etc don't give you a picture of local resource performance, which will most likely be more relevant now with the Silverlight/Flex/RIA movement.  Chrome has Task Manager which is also good for this type of thing but doesn't show disk IO (not to say there isn't an extension for that already).  If you are developing apps, you have to test on IE, period.