Using Meddler to Simulate Web Traffic

  • [アーティクル]

As mentioned back in July, IE8’s new lookahead downloader has a number of bugs which cause it to issue incorrect speculative download requests.

The “BASE Bug” caused the speculative downloader to only respect the <BASE> element for the first speculatively downloaded script file. Subsequent relative SCRIPT SRCs would be combined without respecting the specified BASE, which resulted in spurious requests being sent to the server. Eventually, the main parser would catch up and request the proper URLs, but the spurious requests waste bandwidth and could cause problems for some servers.

When first investigating the speculative downloader problems, I decided to use the Meddler HTTP Traffic Generation tool to build some test cases. Meddler is a simple little tool that allows you to write JavaScript.NET scripts to emulate a web server. Meddler allows for precisely timed delivery of responses, and includes classes to enable basic fuzzing scenarios. The best part of Meddler is that you can use a single MeddlerScript (.ms) file to contain an entire test case, even if that test case is made up of multiple pages, images, scripts, and other resources. These .ms files can be shared with others, run across multiple operating systems, and attached to bugs or test harnesses for future regression testing. The test machine only requires the .NET Framework and Meddler installed, and does not need IIS, Apache, Perl, ASP.NET, etc.

Because the base issue was so simple, I was able to quickly build a simple MeddlerScript which demonstrates the BASE Bug. If you’d like, you can follow along using my MeddlerScript: PreParserBaseBug.ms.

The test script generates the following sample HTML:

<html><head><base href="https://ipv4.fiddler:8088/pass/"></base>
<script type="text/javascript" src="inc/1.js"></script>
<script type="text/javascript" src="inc/2.js"></script>
<script type="text/javascript" src="inc/3.js"></script>
<script type="text/javascript" src="inc/4.js"></script>
<script type="text/javascript" src="inc/5.js"></script>
<script type="text/javascript" src="inc/6.js"></script>
<script type="text/javascript" src="inc/7.js"></script>
<script type="text/javascript" src="inc/8.js"></script>
<script type="text/javascript" src="inc/9.js"></script>
</head>
<body> Test page.</body></html>

Note that I plan to watch the network traffic with Fiddler, and because traffic sent to localhost isn’t proxied, I will use “ipv4.fiddler” as an alias to 127.0.0.1.

When visiting the Meddler test page, the traffic from IE is as follows:

Screenshot of original incorrect network traffic

As you can see, there are spurious download requests containing the wrong path; these are shown in red as the MeddlerScript is designed to return failure for such requests. Later, the correct URLs are downloaded as the main parser encounters the script tags and correctly combines the URLs.

Today's IE8 Cumulative Update (KB974455) fixes the BASE Bug. After installing the update, loading the sample HTML results in no spurious requests-- each script URL is correctly relative to the specified BASE.

Screenshot of corrected network traffic

Please note that while the BASE bug is fixed, the “4k Bug” is not fixed by this update. If you want to view that bug in action, try this script: PreParser4kBug.ms. As it is a timing issue, you may need to reload the “hammer” page a few times to encounter the problem.

While Meddler is rather simplistic, it can be very useful for sharing test cases and simulating the behavior of web servers. You can use Meddler to build reduced test cases that reliably generate problematic HTTP responses.

Until next time,

-Eric

Comments

  • Anonymous
    October 16, 2009
    The comment has been removed

  • Anonymous
    October 23, 2009
    Great post! Great to see the BASE bug fixed. Kudos to the IE team for listening & responding.  

  • Anonymous
    October 30, 2009
    Good to hear. I was wondering if the IETeam would or would not fix the issue (with an IE8 update)... No I've the response [;)]

  • Anonymous
    January 12, 2010
    One update: It turns out that this fix went out for Vista/XP in October but the fix hasn't been released for Win7 at this time. I'm not allowed to make statements about timelines for unreleased fixes, but I will say that we do understand the urgency in getting this fix out.

  • Anonymous
    January 21, 2010
    Note: The BASE issue was fixed for Windows 7 users in today's cumulative update, so the IE8 BASE bug is now resolved on all platforms.

  • Anonymous
    March 31, 2010
    Note: There's still an active bug on the BASE element failing to be respected when the containing page runs in Standards mode and the BASE URI specifies a HREF that uses the FILE:// protocol. This is totally unrelated to the preparser issue discussed in this post.

  • Anonymous
    August 02, 2010
    Today I noticed a strange behavior of my up-to-date IE8 on Win7 applying the <base> to links. When the url in the <base> is "domain.tld" or "sub.domain.tld" everything works fine, but "sub.sub.domain.tld" doesn't work. It works for <script> and <style> though

  • Anonymous
    August 03, 2010
    @Jonny: Please be more specific about what you are trying to say (e.g. provide sample HTML). You can email it to me if that's easier for you.  Thanks.

  • Anonymous
    August 11, 2010
    @Johnny007: You’ll notice that your BASE isn’t respected in Firefox or Chrome either. If you block your script from downloading, you’ll find that your page works. The reason is that your script is rewriting your hyperlinks incorrectly after the page loads.

  • Anonymous
    May 03, 2012
    The comment has been removed