High memory in IIS due to scriptproxy.dll

One of my customers had a problem on his IIS server, where the IIS process hosting his web application was suffering from very high memory utilization over time. Eventually the process got so high in memory that the web application became completely unresponsive and incoming requests were no longer getting served. Based on these details, we began troubleshooting the issue as a memory leak/high memory issue.

The first step was to capture a memory dump of the problematic process, when its memory usage was at its worst point. The customer had actually already captured the memory dump before calling, so we moved to the next step immediately: data analysis.

NOTE: The customer had not used the Debug Diagnostics tool to capture the dump file. The Debug Diagnostics tool is able to capture specific memory leak data which would have made the analysis much easier. However, in this instance we were still able to get to the root cause even without the leak tracking functionality.

The dump files showed the CRuntime heap growing with various allocation sizes and various number of allocations. If we looked specifically at the allocation groups, we could see that some contained data while others contained pointers to objects. Generally, large allocations contained the data (such as a 100k ASP response) and smaller allocations contained pointers (such as a pointer to a script object).

Here is a snippet from one of the larger allocations and the data it contained:

0:000> dc
098b0080 002a002a 002a002a 002a002a 002a002a *.*.*.*.*.*.*.*.
098b0090 002a002a 002a002a 002a002a 002a002a *.*.*.*.*.*.*.*.
098b00a0 002a002a 002a002a 002a002a 002a002a *.*.*.*.*.*.*.*.
098b00b0 002a002a 002a002a 002a002a 002a002a *.*.*.*.*.*.*.*.
098b00c0 002a002a 002a002a 002a002a 002a002a *.*.*.*.*.*.*.*.
098b00d0 002a002a 002a002a 002a002a 000a000d *.*.*.*.*.*.....
098b00e0 00200027 00750050 00700072 0073006f '. .P.u.r.p.o.s.
098b00f0 003a0065 00530009 00740065 00200073 e.:...S.e.t.s. .
0:000>
098b0100 006c0061 0020006c 00740073 006e0061 a.l.l. .s.t.a.n.
098b0110 00610064 00640072 00700020 00670061 d.a.r.d. .p.a.g.
098b0120 00200065 006e0069 006f0066 006d0072 e. .i.n.f.o.r.m.
098b0130 00740061 006f0069 000d006e 0027000a a.t.i.o.n.....'.
098b0140 000a000d 00200027 006e0049 00750070 ....'. .I.n.p.u.
098b0150 00730074 0009003a 006f006e 0065006e t.s.:...n.o.n.e.
098b0160 000a000d 000d0027 0027000a 00520020 ....'.....'. .R.
098b0170 00740065 00720075 0073006e 0009003a e.t.u.r.n.s.:...
0:000>
098b0180 006f006e 0065006e 000a000d 002a0027 n.o.n.e.....'.*.
098b0190 002a002a 002a002a 002a002a 002a002a *.*.*.*.*.*.*.*.
098b01a0 002a002a 002a002a 002a002a 002a002a *.*.*.*.*.*.*.*.
098b01b0 002a002a 002a002a 002a002a 002a002a *.*.*.*.*.*.*.*.
098b01c0 002a002a 002a002a 002a002a 002a002a *.*.*.*.*.*.*.*.
098b01d0 002a002a 002a002a 002a002a 002a002a *.*.*.*.*.*.*.*.
098b01e0 002a002a 002a002a 002a002a 002a002a *.*.*.*.*.*.*.*.
098b01f0 002a002a 002a002a 002a002a 002a002a *.*.*.*.*.*.*.*.
0:000>
098b0200 002a002a 002a002a 002a002a 002a002a *.*.*.*.*.*.*.*.
098b0210 002a002a 002a002a 002a002a 002a002a *.*.*.*.*.*.*.*.
098b0220 000a000d 00090027 0070006f 00690074 ....'...o.p.t.i.
098b0230 006e006f 00650020 00700078 0069006c o.n. .e.x.p.l.i.
098b0240 00690063 000d0074 0072000a 00730065 c.i.t.....r.e.s.
098b0250 006f0070 0073006e 002e0065 00750062 p.o.n.s.e...b.u.
098b0260 00660066 00720065 003d0020 00740020 f.f.e.r. .=. .t.
098b0270 00750072 000d0065 0027000a 00750020 r.u.e.....'. .u.

And here is a sample of one of the smaller allocations (16 bytes each):

030c2f00 18036524 00000000 0315cfd8 00000000 $e..............
030c2f10 00030007 00080101 09c242e0 140bc5b8 .........B......
030c2f20 00000000 00000000 0b3efb7c 00000042 ........|.>.B...
030c2f30 00000005 1c284fc8 a15ac8d4 00000000 .....O(...Z.....
030c2f40 030d7178 52495748 00070039 00080101 xq..HWIR9.......
030c2f50 6b658ba8 6b658b90 6b658b7c 6b658b64 ..ek..ek|.ekd.ek
030c2f60 6b658b4c 6b658b38 6b658b28 6b658af4 L.ek8.ek(.ek..ek
030c2f70 6b658ae4 6b658ad4 6b658ac4 6b658aac ..ek..ek..ek..ek
dc 0x030c33b0+0x8
030c33b8 18036524 00000000 09c31ab8 06a8d500 $e..............
030c33c8 00000000 00000000 00040003 00080101 ................
030c33d8 18036524 00000000 0a09ba90 00000000 $e..............
030c33e8 00030007 00080101 41d8ff20 00000000 ........ ..A....
030c33f8 00000000 00000000 0b62065c 0000004a ........\.b.J...
030c3408 00000000 6bba0068 1a858c47 00000000 ....h..kG.......
030c3418 00000800 00000000 00070005 00080101 ................
030c3428 00000190 00000000 0311a750 00000000 ........P.......
dc 0x030c33d0+0x8
030c33d8 18036524 00000000 0a09ba90 00000000 $e..............
030c33e8 00030007 00080101 41d8ff20 00000000 ........ ..A....
030c33f8 00000000 00000000 0b62065c 0000004a ........\.b.J...
030c3408 00000000 6bba0068 1a858c47 00000000 ....h..kG.......
030c3418 00000800 00000000 00070005 00080101 ................
030c3428 00000190 00000000 0311a750 00000000 ........P.......
030c3438 00000000 030c4f20 00000000 00000000 .... O..........
030c3448 00050003 00080101 18036524 00000000 ........$e......

In the large allocations we could actually see the data in a readable format. It was an ASP script sample, from one of the ASP template objects. But looking at the smaller allocations we saw that the data was in HEX, and there was a repeating 1st parameter of 18036524. This first parameter gave us a good clue as to what allocated the memory; looking at the location that the memory address pointed to, we see it was a function of the component Scriptproxy.dll:

0:000> dds 18036524
18036524 18031000 scriptproxy+0x1000
18036528 180310b0 scriptproxy+0x10b0
1803652c 180310e0 scriptproxy+0x10e0
18036530 180310f0 scriptproxy+0x10f0
18036534 18031160 scriptproxy+0x1160

dc 0x04bccbb0+0x8
04bccbb8 18036cd0 18036cb8 18036ca8 18036c94 .l...l...l...l..
04bccbc8 18036c80 18036c68 05872ad8 05872ad8 .l..hl...*...*..
04bccbd8 05872adc 00000000 00000000 05872ae0 .*...........*..
04bccbe8 00000000 05866218 04bccc00 07e42008 .....b....... ..
04bccbf8 00000000 00000000 00000000 00000000 ................
04bccc08 00000000 00000000 00000000 00000000 ................
04bccc18 00000000 00000000 00000000 00000000 ................
04bccc28 00000000 00000000 00000000 00000000 ................
dc 0x04bccfc8+0x8
04bccfd0 18036cd0 18036cb8 18036ca8 18036c94 .l...l...l...l..
04bccfe0 18036c80 18036c68 05872910 05872910 .l..hl...)...)..
04bccff0 05872914 00000000 00000000 00000000 .)..............
04bcd000 00000000 07dd5398 626f2000 04bd0cf0 .....S... ob....
04bcd010 00000000 00000000 00000000 00000000 ................
04bcd020 00000000 00000000 00000000 00000000 ................
04bcd030 00000000 00000000 00000000 00000000 ................
04bcd040 00000000 00000000 00000000 00000000 ................

:000> dds 18036cd0
18036cd0 18031f30 scriptproxy!DllGetClassObject+0x850
18036cd4 180320f0 scriptproxy!DllGetClassObject+0xa10
18036cd8 18032590 scriptproxy!DllGetClassObject+0xeb0
18036cdc 18032110 scriptproxy!DllGetClassObject+0xa30
18036ce0 18032130 scriptproxy!DllGetClassObject+0xa50
18036ce4 18032150 scriptproxy!DllGetClassObject+0xa70
18036ce8 18032170 scriptproxy!DllGetClassObject+0xa90
18036cec 18032190 scriptproxy!DllGetClassObject+0xab0
18036cf0 180321b0 scriptproxy!DllGetClassObject+0xad0
18036cf4 180321d0 scriptproxy!DllGetClassObject+0xaf0
18036cf8 18032200 scriptproxy!DllGetClassObject+0xb20
18036cfc 18032220 scriptproxy!DllGetClassObject+0xb40
18036d00 18032240 scriptproxy!DllGetClassObject+0xb60
18036d04 18032260 scriptproxy!DllGetClassObject+0xb80
18036d08 18032280 scriptproxy!DllGetClassObject+0xba0
18036d0c 180322a0 scriptproxy!DllGetClassObject+0xbc0

From this data we could see that a component called scriptproxy.dll appeared to be allocating memory to store instances of its class object, and appeared to be populating its class objects with the ASP template data. Unfortunately, the memory it was allocating was not ever being released, which was resulting in the high memory.

Because Scriptproxy.dll is not a Microsoft component, we were not able to discern from the memory dump why it was acting this way, nor whether this was expected behavior. The easiest was to resolve the customer’s high memory problem quickly was to remove the Scriptproxy.dll component and disable it from loading in the IIS processes. Once the customer did that, the memory problem no longer occurred. I then suggested to the customer that they follow up with the vendor of Scriptproxy.dll for further assistance, since really they would be the experts and would know why Scriptproxy.dll was behaving this way. The lm command inside the windbg debugger can be used to show details about a component, so I ran it against Scriptproxy to get the vendor info:

0:000> lmvm scriptproxy
start end module name
18030000 1803b000 scriptproxy (deferred)
Image path: C:\Program Files\Network Associates\VirusScan\scriptproxy.dll
Image name: scriptproxy.dll
Timestamp: Thu Nov 11 12:31:26 2004 (4193CC1E)
CheckSum: 00000000
ImageSize: 0000B000
File version: 8.0.0.955
Product version: 8.0.0.0
File flags: 0 (Mask 3F)
File OS: 40004 NT Win32
File type: 2.0 Dll
File date: 00000000.00000000
Translations: 0000.04b0
CompanyName: Network Associates, Inc.
ProductName: VirusScan Enterprise
ProductVersion: 8.0.0
FileVersion: 8.0.0.955
PrivateBuild: VSE.8.0.0.955 F1
FileDescription: VirusScan Script Proxy
LegalCopyright: Copyright© 1995-2004 Networks Associates Technology, Inc. All
Rights Reserved.

My customer did get back to me after speaking with the vendor, and it turns out that they had seen these types of issues before with Scriptproxy when it was run on server application platforms that relied on script processing.  They also pointed to some publicly available documentation, https://kc.mcafee.com/corporate/index?page=content&id=KB55963, to assist our customer further.