Buffer Overrun Detected while installing - SQL Server 2000 SP4

  • Article

Understanding the functionality:

- When the domain authentication details are provided - in code it looks for the PDC.

- A function by name VerifyPDC is invoked to verify the PDC

- A parameter is passed to the VerifyPDC function that can hold a maximum of MAX_COMPUTERNAME_LENGTH+1, which is as below:

 max 15 characters of PDC name

and a Null terminator (\n)

- This function invokes another function by name DCGetDcName - this will return the name of the PDC.

- In the function DCGetDcName they were padding '\\' to the PDC name.

- So for example if the DC name is 13 characters in length, the computation goes as below:

      13 (length of Domain controller name)

        1 (for null terminator)

        2 (for \\)

     -------

      16 (total of 16 characters can be stored in the stack for that variable)

- If we exceed then the buffer overflow error is expected.

General understanding:

1. Issue occurs when we are upgrading to SP4 on the cluster. When DC's are installed on both the nodes of the cluster and Domain controllers having more than 14 or 15 characters.

2. The workaround specified in the bug - Will work till SP3:

Virtual server setup crashes if the computer name of PDC of the domain contains 14 or 15 characters

The workaround is to use debug version of sqlsui.dll and sqlsut.dll and replace them with the (sqlsui.dll & sqlsut.dll) files.

3. The way it works till SP3 is...

- The variables in debugger version was created with some extra space so it was capable of storing the extra characters.

4. However this workaround doesnt help in SP4. In SP4 they have introduced a new switch.

5. The functionality of this switch is to throw an exception if there is any stack overflow and the exception is handled and it gives the following error message:

"Buffer Overflow detected"

6. This functionality has been added to avoid any malicious code being executed.

7. In this case, the computer name is 14 characters in length and in the code is padding '\\' and adding the \n terminator reaches to 17 characters which is out of the boundary of stack -

so it generates the above error.

8. The recommendation to fix the issue is - to rename the domain controller name with max of 13 characters in length.