A special GNE (General Network Error) messages when running SQL Server after installing service pack 1 for Windows Server 2003 and TCP registry key SynAttackProtect

  • Article

The client error messages related to this post are the followings.

For MDAC client,

[DBNETLIB]General network error. Check your network documentation.

[DBNETLIB]ConnectionRead (recv()). For SNAC client,

For SNAC client,

“TCP Provider: An existing connection was forcibly closed by the remote host”.

We found this issue after SQL Server 2005 Beta2 when w2k3sp1 (Windows 2003 server service pack 1) was one of our default OS for test. Lately, quite a few customers have reported seeing this issue during connection scalability test of SQL Server 2005 or on SQL Server 2000 production server, after upgrade base OS to w2k3sp1. This issue results from a subtle winsock behavior change caused by that w2k3sp1 server has enabled SynAttackProtect by default. This key is configurable through “HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesTcpipParametersSynAttackProtect{DWORD}”. As documented in SQL Server 2005 Book-On-Line, one way to resolve this issue is to disable it by set SynAttackProtect value to 0. Note that, event though we document this solution in SQL Server 2005 BOL, it applies to both SQL Server 2005 and SQL Server 2000. In this post, I want to explain what is really happening under the cover and help you to understand the risk to turn this key off.

What is the Sync Flood Attack?

Sync-Flood-Attack is a network Deny-Of-Service attack. It takes advantage of TCP connection handshake protocol by sending large amount of TCP Sync packets to a victim server with spoofed IP source address. In standard TCP implementation, once a new sync packet is received, a half opened connection handle and its associated context is allocated and queued in corresponding TCP listening socket backlog queue. With combination of high frequency of sync packet arrivals and the default timeout value of half open connection entries, the backlog queue, max to 200 by default on w2k3 standard server, can easily be exhausted; as result, sync flood attack can cause server become non-responsive to normal connection requests. Here is some more info, https://www.microsoft.com/technet/community/columns/cableguy/cg1204.mspx.

A solution to Sync Flood Attack

To mitigate such issue, a solution is to hold the half opened connection handle in a cache and only allocate full TCP connection context and en-queue the handle when a connection is fully established after TCP three way handshakes. The cache implemented in Windows is based on a hash table that can virtually hold unlimited half-opened connections. The cache also uses smaller timeout value for connection entries that stay in the cache so that unresponsive half open connection entry will be evicted faster.

The side effect to the normal connections

For normal connections, without SynAttackProtect, as soon as the backlog queue is full, server side TCP will send TCP reset in response to new TCP sync packet. The client side winsock connect() call that send first TCP sync packet to the serverl will receive the TCP reset and fail with ERROR_CONNECTION_REFUSED. When SynAttackProtect is on, even the backlog queue is full, a new connection request arrived at the server will still be accepted because the half connection cache is visually unlimited. However, once the handshake succeeds, the connection can still be rejected because the server TCP find out that the backlog queue is actually full when moves its entry from the cache to the backlog queue. With the timing shifted to send TCP reset from during the connection establishment handshakes to after, the client succeeds when calling winsock connect() but fails with WSAECONNRESET when performing the first recv() or send() on established connections.

What is the impact to SQL Server Client Driver?

SQL Server client drivers, including MDAC and SNAC, have retry logic for failure when connect() fails, a logic to improve user experience when there is transient network failure. However, they do not retry when first recv() or send() fail because of a TDS protocol compatibility issue with older versions of SQL server such as 7.0. With SyncAttackProtect enabled, due to the timing shifted, more first recv() or send() are failing inside the SQL Server client driver; and thus, more GNEs are raised to applications.

Solutions to this GNE issue.

As mentioned above, server can disable the SynAttackProtect in w2k3sp1. But keep in mind this solution makes the server subject to sync flood attack if it is open to public access. Another solution is to add a retry-logic to the SQL server client application to alleviate the problem. I am glad some applications are already doing that. Database application developers also need to keep in mind that the retry logic should be designed in such a way not to overwhelm the server.

 

Do you know that you can post question w.r.t SQL Server data access, connectivty issues at https://forums.microsoft.com/MSDN/ShowForum.aspx?ForumID=87&SiteID=1 ?

Nan Tu

Software Design Engineer, SQL Server Protocols

Disclaimer: This posting is provided "AS IS" with no warranties, and confers no rights

Comments

  • Anonymous
    June 02, 2006
    One way to help users to diagnose this kind of problem is to be able to demonstrate it and exacerbate it under test conditions.

    Can you cobble together a simple script that will stress the SynAttack logic to be able to demonstrate the problem with a live application. This test can then be included with some UAT acceptance testing of applications.

    Thanks

  • Anonymous
    December 04, 2006
    This is an awesome blog posting. Unfortunately, I think you got the registry key wrong... It should be SynAttackProtect not SycAttackProtect http://technet2.microsoft.com/WindowsServer/en/Library/8d3a9f4d-13d1-4280-ac57-30242504d8ba1033.mspx Daern

  • Anonymous
    December 04, 2006
    O problema afeta as versões 2000 e 2005 do SQL Server, quando este roda em um servidor Windows...

  • Anonymous
    March 24, 2007
    Arghh, what are these "General Network Error" messages? You know the ones, that randomly with...

  • Anonymous
    March 26, 2007
    Thanks Daern for the correction on a typo.

  • Anonymous
    March 26, 2007
    Peterson, The GNE is the type of error message with MDAC. With SNAC and ADO.net2.0, the error messages are improved and lots of issues can be identified much faster and easier. If you see GNE issues, please post your finding here or on the forum, at http://forums.microsoft.com/MSDN/ShowForum.aspx?ForumID=87&SiteID=1 Thanks,

  • Anonymous
    April 16, 2007
    Hi there, I'm facing the same problem. When i make changes to the registry, it resolved the problem at internal environment. However changes on live environment had not effect. The error still occur. Any ideas what else is missed?

  • Anonymous
    May 22, 2007
    Hi All, I am getting the same error message([DBNETLIB]General network error. Check your network documentation). We are using MS SQL 2000 but the operating system is Windows Server 2000 SP2. Can anybody tell me if the reason of this problem on Windows 2000 server is also same or can it be due to some other reason? Thanks in advance Vineet

  • Anonymous
    July 15, 2007
    "General Network error .........." This error message really got me worked, but fortunately i was able to find the reason. To replicate the error you need to have SQL Server Management Studio (2005) may be with SQL Server 2000 it is Enterprise Manager, but not MSDE. You may create a small VB.NET app using the following code Dim conn as SqlConnection Dim cmd as SqlCommand ' Remember to change values between [] Try    conn = New SqlConnection("Data Source=" + [Your Server Name] + ";Integrated Security=false; User ID=" + [Your User Name] + ";Password=" + "Your Password" + ";")    conn.Open()    cmd = New SqlCommand    cmd.Connection = conn    cmd.CommandText = "CREATE DATABASE [aftermarket]"    cmd.ExecuteNonQuery() Catch ex As SqlException    Console.WriteLine(ex.Message)    ' OR following if under Winforms    MessageBox.Show(ex.Message) Finally    conn.Close() End Try OR You may create a small C# app using the following code SqlConnection conn; SqlCommand cmd ; try {    // Remember to change values between []    conn = new SqlConnection("Data Source=" + [Your Server Name] + ";Integrated Security=false; User ID=" + [Your User Name] + ";Password=" + "Your Password" + ";");    conn.Open();    cmd = new SqlCommand();    cmd.Connection = conn;    cmd.CommandText = "CREATE DATABASE [MyDatabase]";    cmd.ExecuteNonQuery(); } catch (SqlException ex) {    Console.WriteLine(ex.Message);    // OR following if under Winforms    MessageBox.Show(ex.Message); } finally {    conn.Close(); } Compile and run the app, hopefully a Database named "MyDatabase" will be created Now start SQL Server Management Studio (for 2005) or Enterprise Manager (for 2000) and delete the Database just created, now run the app created earlier, you might get the dreaded error message, try to run the app atleast 3 times you will get the same error message. Now question is , how do i correct the problem, simple - Reboot the system and try running the app again. Indeed a painful way of overcomming an error message. This error message ONLY show up when you have, and has no connection with the Timeout property. Integrated Security=false User ID=[Your User Name] Password=[Your Password] Note: This is Microsofts general-purpose error message and this solution may not work in all cases. Barretto

  • Anonymous
    November 13, 2007
    I get the following error msg while uploading a database to our corporate office's offline database server running MS SQL server 2000. The corporate office has to manually upload the data from their server itself. This error has been raised since our VSAT provider updated their VSAT configuration (NET modem).

  • Anonymous
    November 21, 2007
    hello respected sir, I am sachin lad from mumbai. I am currently working in one of the software company. I have getting one error while running the application on emulator.my operating system is win xp and sql server is threre. please reply me as early as possible.

  • Anonymous
    June 23, 2008
    I saw this same error with SQL Server 2005.  After much thrashing, I found that the problem was being caused by changes to the TCP/IP registry parameters, specifically TcpMaxConnectRetransmissions and TcpMaxDataRetransmissions, both of which set to 0.  Changing these values both to 2 solved the database problem. Note: These parameters were originally set to 0 for good reason: to prevent unreasonably long timeouts when trying to access disconnected network devices.  And it makes no sense that SQL Server would need to retry in this case, since the client application is on the same machine as the server.  But it is what it is, so I have to sacrifice performance of my network data collection application to get SQL Server to work at all.

  • Anonymous
    July 28, 2008
    I'm using DTS to upload data from text file to sql 2000 database,When i'm loading 70000 records its work fine,but when i'm trying to load more than 70000(3 Million Records) I'm getting this error message, please help."[DBNETLIB][ConnectionRead(recv()).]General Network error.Check your network documantation."

  • Anonymous
    November 12, 2008
    I got the error message, and we resolved it by changing the "Enable NetBIOS over TCP/IP" setting radio button under the "WINS" tab in the Advanced TCP/IP settings.  Also, be sure and check your binding order and NIC cards under Device Manager: we had one disabled.

  • Anonymous
    March 19, 2009
    in my production server. I got the error message. But when I upgrade NTWDBLIB.DLL from 7.0 to 8.0. No more the error message appeared.

  • Anonymous
    May 28, 2009
    The comment has been removed

  • Anonymous
    May 28, 2009
    one more: on clients are Bussines security (nod32 with FW), i dont know how to disable this fw or add some exception?

  • Anonymous
    October 22, 2009
    Check your NIC and upgrade it, and take a look at http://support.microsoft.com/kb/942861

  • Anonymous
    March 06, 2011
    i'm getting general network error while accessing application from server. server machine: windows server 2008 standard client machine: windows 7 database: sql server 2005 standard edition switch : 24 port unmanaged switch network cable: CAT-5 cable. can anyone help me......