IIS7. Why Global Managed Modules are Disallowed

  • Article

Ok, I have been pretty busy the past 10 days with both work and life, and I have not had a chance to post any blog entries. While I cannot say that I am any less busy now, I definitely have a lot of catchup and posts to make... so here goes.

This question was originally posted in the microsoft.public.platformsdk.internet.server.isapi-dev newsgroup on the msnews.microsoft.com NNTP server. Since all the technical previews at events like TechEd and PDC will be high-level concepts/details, the astute reader will always have some unanswered questions.

Now, while I cannot promise that I will answer everything about IIS7, nor can I promise that my answers will forever be correct (remember, we are talking about IIS7, an unreleased beta product), I will try to give as much currently-correct technical information as I can.

Question:

Hi!

I know this is kinda not the right group for this question, but if you know one that will be better suited, please let me know... That aside, though:

I have written a couple of ISAPI filters for our company, and they work just fine. Seeing the changes being made in IIS7, though, I thought I better get familiar with HTTPModules :-). So I just wrote an HTTPModule for IIS 6, using C#. This of course utilizes the old (current) architecture, and I'm somewhat confused about how the ISAPI callback functions "map" to the HTTPModule members.

In my current ISAPI filter, I reserve a piece of memory in GetFilterVersion, let's say 5 MB. GetFilterVersion was called on every load of the filter, so once per worker process; this means I have [#w3wp.exe} times 5 MB allocated. If I understand the HTTPModule structure right, the best match equivalent to GetFilterVersion would be the constructor, or possibly the Init() member, so the logical thing to do would be to "allocate" (as far as .NET lets me) my memory there now. There's quita a problem with this though...

As I saw it, the constructor of my module will be called once for every ASP.NET application that is being requested, and this gives me some headaches. In my case, we have about 50 worker process on the server, serving about 2500 websites. Each website has at least one appliaction, possibly more. Of course they're not all running at the same time, but still: We have a whole lot more of applications than we have worker processes. Let's just say that 800 ASP.NET appliactions are running at the time, so this would mean I'd end up with using 800 x 5 MB of memory. Definitely not desirable... The fact that I can't deliberately free my memory makes this even worse, but that's a problem I always struggle with in C# ;-)

So, is this "once per application" problem that I have now just an ASP.NET thing, because the module still lives inside the aspnet_isapi.dll? Will the behaviour be different in IIS7, and will there be a place that's only called once per w3wp.exe startup?

Thanks in advance,

Answer:

Since IIS7 is not available publicly and the API is beta (i.e. it is subject to change, and I know that it must have changed significantly since your point of reference...), I really would not spend TOO much time thinking about the details yet...  ;-) But since you are interested, this is a perfectly fine forum to ask.

Based on your line of questioning, the main points I want to emphasize are that:

  1. IIS7 functionality can be extended using native and managed code.
  2. Functionally speaking, neither extensibility point is a strict subset of the other. They each have abilities that are lacking in the other. They also share a lot of common functionality of the IIS server core.
  3. There is no direct mapping between any the following "legacy" concepts of ISAPI Filter, ISAPI Extension, ScriptMap Engine, HttpModule, and HttpHandler with the new IIS7 concepts of Global Modules, Modules (i.e. per-app capability selection list), and Handlers, though there are many conceptual similarities.
  4. Consequently, programming language remains non-negotiable. You need to choose the abilities that you need, then choose the necessary extensibility point (native code or managed code), and finally the programming language necessary for the task.

For example, I would not assume that things in IIS6 such as ISAPI Filter, ISAPI Extension, and CGI automatically transform into an HttpModule or HttpHandler in IIS7 because they definitely do not. If you think about it a little, you should realize that something like a global ISAPI Filter which triggers on every single request can never map to an HttpModule. Security considerations alone should be reason enough.

Now, in your case, I think your design requires a native code global module because it has the behavior that you need. Here is the crux of what is going on:

Conceptually, you can view IIS7 as a componentization of the IIS6 server core accomplished by:

  1. Introducing a single, new C-based API which uses pure native code. Code which implements this API are called global modules.
  2. Architecting a new server core engine which consumes .config configuration and exposes the new API from #1 as native extensibility mechanism.
  3. Rewiring all IIS6 features in terms of the new API from #1. This includes legacy extensibility mechanisms like ISAPI Extension, ISAPI Filter, and CGI as well as basic server behavior like Anonymous/Basic/Integrated Authentication, Default Document/Directory Browsing, Static File Handler, Custom Error Handler, etc. Presently, IIS7 consists of about 40+ such global modules.
  4. Rewire ASP.Net in terms of the new API from #1. Basically, move the .Net AppDomain activation code from ASPNET_ISAPI.DLL into a new global module that directly implements the new API from #1 and bypasses the entire ISAPI Extension legacy compat layer of #3. The resulting ASP.Net implementation still exposes the HttpHandler and HttpModule interfaces but has deeper access to the IIS7 request pipeline since it shims directly to the native global module instead of indirectly through an ISAPI extension API of IIS6.

At this point, the astute reader should be wondering why global modules MUST be in native code and not managed code. After all, the managed code in an HttpModule now has direct access to the native IIS7 request pipeline through the ASP.Net global module "shim" that is provided out of the box, and since the global module can execute managed code at any point, it *should* be able to run an HttpModule "globally".

Well, yes, that idea is possible, but after long design considerations, we decided to stick with the ASP.Net paradigm of application isolation through AppDomains for managed applications, so there is no such thing as "global" managed code. The security, performance, and complexity ramifications of global managed modules were just too problematic. For example, here are some of our thoughts and counterpoints:

  • Suppose IIS7 hosts only one AppDomain, the "global" AppDomain, for all managed applications so that defined HttpModules and HttpHandlers are "global" by definition. This violates the security model of every current IIS server that has >1 ASP.Net app and is not scalable nor performant.
  • Suppose IIS7 hosts multiple AppDomains and a single "global" AppDomain where global HttpModules and HttpHandlers are loaded. Now, this AppDomain is located in some dedicated process, which means that all requests filtered by global modules in managed code requires a double hop (a la IIS5 process model) and makes the dedicated process a singular point of failure, destroying the IIS Worker Process Model.
  • Suppose IIS7 hosts multiple "global" AppDomains which are distributed per Application Pool. Now, how should the supposedly singular "global HttpModule" share copies of its data structure between all processes of all Application Pools, keep them synchronized without any user intervention (remember, HttpModule authors have no idea of the process model ramifications and would not be locking anything), and do this automatically, performantly, and securely? Is it even desirable?

In the end, we decided that the implementation cost, performance penalty, and the resulting security ramifications were simply too complex. There are good situations for both native and managed modules, and we attempted to strike a reasonable balance between functionality and ease. Allowing managed code a chance to manipulate the incoming request URL/headers, participate in request authentication/authorization inline, and perform caching/logging logic cover most users' needs. Besides, we want global module developers to be a little more "responsible" since it carries huge consequences on server behavior... so native code serves to weed that out a bit. :-)

Now, if you have a bright idea on how to implement global managed modules in a secure, performant, reliable, and functional manner without destroying basic IIS tenets and remain easy to use, feel free to share your thoughts... :-) Until then, global managed modules are not allowed in IIS7.

//David

Comments

  • Anonymous
    September 13, 2005
    Hi!

    (I'm the original poster quoted above :))

    I understand that all the points you make at the end are real show-stoppers for implementing managed global modules, but the reason they are show-stopperes are not because of the managed code, but rather because of this decision:

    "[..] but after long design considerations, we decided to stick with the ASP.Net paradigm of application isolation through AppDomains for managed applications [..]"

    If a managed global module would not be confined to an AppDomain, none of the above mentioned problems would exist. I don't exactly understand why that is necessary.
    Is there a reason why you can't just build a managed wrapper around the C-API that all the
    native global modules are based on?
    I mean, it doesn't have to be shaped after IHTTPModule, and doesn't have to utilize any ASP.NET functionality. Just make it look and feel like the C-API, just in managed code.
    Would that be a possibility?

    Ciao,

    Thomas

  • Anonymous
    September 14, 2005
    Thomas - Sorry, but a managed wrapper of the native code API only serves to introduce another managed extensibility API in addition to HttpModule/HttpHandler, and that is undesirable for many reasons:
    1. API Confusion. I am extending IIS7 with managed code , so what do I write? Behaviorally they will be very different.
    2. Cost. It is simply a duplicate interface for us to implement, test, and maintain. This cost must have a clear benefit
    3. Zero benefit. No clear benefit other than developer laziness, and that is not something we want to encourage with an API.

    And if you say that you have a bunch of C# programmers and really want to extend IIS7 with a global module but cannot do it now, then I say "start getting acquainted with C". :-)

    //David

  • Anonymous
    September 14, 2005
    "And if you say that you have a bunch of C# programmers and really want to extend IIS7 with a global module but cannot do it now, then I say "start getting acquainted with C". :-)"

    I am very glad you said that, and I'll make sure to let everyone here know :-D

  • Anonymous
    September 14, 2005
    Thomas - Good to hear.

    In my mind, the choice of programming language should not be a concern, and here is why. The new API for writing global modules should be nowhere near as unfriendly as ISAPI Filter / Extension of the past.

    Yes, the new API is far larger and more complex than ISAPI can ever be, but this time around several key things will be different that I think should make it more palatable to more people, both experienced and novice.

    I took all the feedback from my own experiences learning and working with ISAPI as well as what I observed in MANY other people using ISAPI, and I made sure the new API addressed those issues so that you should be able to focus more on your task than wrestling with the API.

    I also plan on providing many complete sample modules with full source code and detailed walk-through comments embedded (some may be my own dev/test modules...) showing how to utilize just about every aspect of the new API.

    As always, I will be looking at how to gather and grow a community of IIS module developers so that we can build modules the right way with solid code, share them with everyone to help their efforts, and hopefully accumulate a good selection of modules ourselves. It will probably be modeled after WiX on SourceForge and will be Open Source.

    And you can expect that this blog will continue to be a source of insightful information about IIS including IIS7...

    So, I think that all these changes should ease the task of module development significantly to the point that programming language should no longer be a barrier (however, having a brain and human intelligence is still required. ;-) )

    //David

  • Anonymous
    September 16, 2005
    Hi!

    I want know, the IIS 7.0 will support rewrite?

    Thank you!

    Cmb.

  • Anonymous
    September 16, 2005
    Cmd - Of course IIS7 will support rewrite. So does IIS6 and every earlier version of IIS in use today.

    The only question is a matter of availability and cost.

    Nothing stops anyone from programming URL rewriting and contributing it to the IIS community - in fact, there is one such project I was made aware of recently - http://www.iismods.com

    The ISAPI Filter should function on IIS7 as well as any prior version of IIS (if it does what I think it does [I need to code-review to make a definitive statement]).

    //David

  • Anonymous
    September 18, 2005
    eh...
    My meaning is "the IIS 7.0 will include Rewrite support Module in default?"
    But, now I knew.

    I think, The IISmods project is one valuably project, because it's free and opensource.

    Thanks your reply.

    BTW: My name is "Cmb", not "Cmd". :-)

  • Anonymous
    September 18, 2005
    Cmb - Unfortunately, I can neither confirm nor deny whether a rewrite module will exist by default in IIS7.

    I can only say that support for such a module exists in IIS7 as well as previous IIS versions. The only question is a matter of availability and cost.

    BTW, this is the exact same stance that Apache/ASF takes with mod_rewrite. mod_rewrite is not a part of Apache core; it is an add-on module, donated to Open Source, and distros often bundle them together. If no one donated the module nor maintained it, rewrite support would not exist in Apache.

    So, same goes with IIS... it is EVERYONE'S responsibility to maintain a healthy community around extending IIS functionality and sharing it with others. You should not become complacent and wait for Microsoft to introduce/support anything. The APIs are all openly available and documented; go for it.

    //David

  • Anonymous
    September 19, 2005
    Ok, now I knew.
    Thanks your reply.
    I hope the Rewrite module will free. :-)

    Cmb

  • Anonymous
    September 19, 2005
    Cmb - Well, modules are only as free as you make it. It is all software, after all. I am looking forward to helping collaborations that produce IIS modules succeed.

    //David

  • Anonymous
    September 20, 2005
    David,
    eh. right, It's a good idea if i can make it! but i can't, so I will paid if I need run rewrite in IIS and the modules need pays.

    and hope the IIS 7.0 will become a best server-ware to get beyond the Apache.

  • Anonymous
    March 15, 2006
    Lately, I have received quite a few comments concerning a crash in WebEngine.dll when you recycle...

  • Anonymous
    March 19, 2006
    One of the cooler but definitely unsung feature of the IIS7 Server Core is PreConditions. So, that's...

  • Anonymous
    July 18, 2007
    I'm looking forward to running IIS 7 but... I understand the details with the mod_rewrite and Apache, but regardless of why, it IS usually included in distros so it is standard,  easy to use and easy to find support for it. With IIS not being open source, It just seems like MS would be able to include the functionality so that there would be a standard for developers to build from. IIS needs to make it easier for people to set up and run things like Wordpress and Joomla with SEO capability that requires rewrite functionality. Something as easy as "add this to your .hhaccess file"

  • Anonymous
    July 20, 2007
    Ken - Actually, I believe Open/Closed source has nothing to do with what you are describing. IIS users tend to be business oriented who are not incented to share solutions (at least, not supported solutions). Lots of individual efforts, never a community. Apache users tend to be two groups - techies that understand and benefit from sharing code as a community, and a vast majority of everyone else that just stand on the work of those techies and have the luxury of choosing not to share. What is common between them is that users all tend to shy away from taking responsibility (i.e. supporting software) without compensation. Open Source makes this implicit by saying "you've got the source code -- figure it out yourself -- I owe you no support". It is just a different user culture. To me, the question is not for "IIS to make application X easier to run on IIS" because that list is infinite. The question is whether the user culture on IIS supports a particular application to attract the developer to build on it. Because once an application decides to support a platform, it will be as easy as "add this to your .htaccess file". Until the application supports the platform, it will always be a hassle. I believe that IIS7 has all the potential to be a great platform to build a community around, but it faces challenges in that its users are not used to community like this. Someone has to make a major effort to contribute free code like mod_rewrite, mod_proxy, etc to spark community and sharing. //David