ASMX/WS/WCF Web Service: System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host

  • Article

Issue:
Intermittent Socket exception seen on client application trying to fetch data from MS web services.

Troubleshooting:
I recommend collecting application level traces to collect the stack trace information. In addition we can collect the System.Net traces or memory dumps on specific exceptions.

Detailed stack from dump:
0:000> !dumpstack
OS Thread Id: 0x1708 (0)
Current frame: KERNELBASE!RaiseException+0x58
ChildEBP RetAddr Caller, Callee
002dd7fc 7503c54f KERNELBASE!RaiseException+0x58, calling ntdll!RtlRaiseException
002dd82c 72440769 (MethodDesc 7228a444 +0x401 System.Net.ConnectStream.CloseInternal(Boolean, Boolean)), calling (MethodDesc 722f5b60 +0 System.Net.ConnectStream.CallDone(System.Net.ConnectionReturnResult))
002dd844 7402a769 clr!RaiseTheExceptionInternalOnly+0x27c, calling kernel32!RaiseExceptionStub
002dd8e0 7402af54 clr!IL_Throw+0x138, calling clr!RaiseTheExceptionInternalOnly
002dd944 7402aea1 clr!IL_Throw+0x3c, calling clr!LazyMachStateCaptureState
002dd94c 7402c4a4 clr!IL_Rethrow+0x27, calling clr!LazyMachStateCaptureState
002dd950 72887df0 (MethodDesc 722f5ca4 +0x140 System.Net.ConnectStream.IOError(System.Exception, Boolean)), calling (MethodDesc 722f5b60 +0 System.Net.ConnectStream.CallDone(System.Net.ConnectionReturnResult))
002dd970 723f6845 (MethodDesc 722f5bb4 +0xc5 System.Net.ConnectStream.ReadWithoutValidation(Byte[], Int32, Int32, Boolean)), calling clr!IL_Rethrow
002dd9a8 723f6726 (MethodDesc 7228a3dc +0x136 System.Net.ConnectStream.Read(Byte[], Int32, Int32)), calling clr!IL_Throw
002ddaa0 73018a56 (MethodDesc 72d67e88 +0xe2 System.IO.StreamReader.ReadBuffer(Char[], Int32, Int32, Boolean ByRef))
002ddac0 7301893b (MethodDesc 72d67e34 +0x7b System.IO.StreamReader.Read(Char[], Int32, Int32)), calling (MethodDesc 72d67e88 +0 System.IO.StreamReader.ReadBuffer(Char[], Int32, Int32, Boolean ByRef))
002ddae8 7010c168 (MethodDesc 70069474 +0xe8 System.Xml.XmlTextReaderImpl.ReadData())

The code, clearly indicates that we are indeed running in a catch block and thus error is thrown.
https://referencesource.microsoft.com/\#System/net/System/Net/\_ConnectStream.cs,17c93c1ab8aca75d

To mitigate the issue, following workaround is suggested (please review with your network admin before trying them):

Command to disable SNP features :
netsh interface tcp set global chimney=disabled
netsh interface tcp set global rss=disabled
netsh int tcp set global autotuning=disabled
netsh int tcp set global congestion=none
netsh int tcp set global netdma=Disabled

For disabling the offloading:
Open NCPA.cpl on command prompt and open NIC properties and disable the following features :
ARP offload -----Disable
Jumbo packet -----Disable
ipv4 Checksum Offload -----Disable
Jumbo packet -----Disable
Large send Offload V2 (ipv4) -----Disable
Large send Offload V2 (ipv6) -----Disable
NS offload- ----Disable
TCP Checksum Offload (ipv4) -----Disable
TCP Checksum Offload (ipv6) -----Disable
UDP Checksum Offload (ipv4) -----Disable
UDP Checksum Offload (Ipv6) -----Disable

References:
blogs.technet.com/b/onthewire/archive/2014/01/21/tcp-offloading-chimney-amp-rss-what-is-it-and-should-i-disable-it.aspx
https://support.microsoft.com/en-us/help/951037/information-about-the-tcp-chimney-offload,-receive-side-scaling,-and-network-direct-memory-access-features-in-windows-server-2008

Hope this helps!

Thanks
Saurabh Somani