BizTalk gets "General Network Error" under heavy load

  • [アーティクル]

This is a fairly common scenario after Windows 2003 SP1 (or greater) is installed, so I thought I better put in my two cents

Scenario: BizTalk seems to lose connectivity to the MsgBox (and all other databases on that SQL instance)

Problem:  The operating system (w2k3 sp1) running the SQL server instance is misinterpreting the heavy TCPIP connection traffic from the BizTalk App server as a Denial of Service (DOS) attack.

Clarification: This is an operating system level issue. This is not BizTalk or SQL per se (though they are common victims of this security feature). This is a security feature for the OS that affects how the OS interprets large amounts of incoming connections from individual clients.

Factors:

1. Windows Server 2003 SP1 (or higher) is installed on the SQL server hosting the BizTalk DBs

2. SynAttackProtect is still set to “1” on the SQL server

3. Concurrent TCPIP connection requests from a specific BizTalk app server exceeds the critical threshold (Sorry, don’t know what that threshold is!)

Diagnosing:

  1. Look for the following event entries:

Event Type: Error
Event Source: BizTalk Server 2004
Event Category: (1)
Event ID: 6913
Computer: %ServerNameHere%

Description: SQLServer, BizTalkMsgBoxDb, [DBNETLIB][ConnectionWrite (send()).]General network error. Check your network documentation..

Event Type: Warning
Event Source: BizTalk Server 2004
Event Category: (1)
Event ID: 5410
Computer: %ServerNameHere%

Description: An error has occurred that requires the BizTalk service to terminate. The most common causes are an unexpected out of memory error and an inability to connect or a loss of connectivity to one of the BizTalk databases. The service will shutdown and auto-restart in 1 minute. If the problematic database remains unavailable, this cycle will repeat.

Error message: [DBNETLIB][ConnectionWrite (send()).]General network error. Check your network documentation.

Error source:
BizTalk host name: BizTalkHost
BizTalk Windows service name: BTSSvc$BizTalkHost.

Fixes:

1. Set the SynAttackProtect registry setting as described in https://support.microsoft.com/kb/899599

2. ALSO take a look at the Group policy for that SQL server to be sure that the Group Policy does not periodically UNDO the setting change that you did in step 1. I have been in the situation where I fixed the issue, but then the GP Update or reboot undid the fix! (Put the server in a separate OU if you have to, but make sure you isolate the box from policies that would reset this registry key)

Limitations

1. I don’t know how to disable this feature for a specific set of clients (i.e. SynAttackProtect is off for the BizTalk app servers but remains on for other servers). The setting seems to be at the protocol level so I wouldn’t even know where to start looking for network set definition here.

Is there something that I can do to make this blog more helpful to you? I welcome your feedback!

Comments