Override the hardcoded LDAP Query limits introduced in Windows Server 2008 and Windows Server 2008 R2

  • Article

Hello Everyone:

This is Qasim Zaidi. First of all, welcome to my blog site.

Next, I am writing this blog (thanks to a colleague) since some of my customers are running into LDAP limitations which are now hardcoded in Windows Server 2008 and Windows Server 2008 R2. Though, ideally we would like to modify the paged queries but depending on the number of applications a customer might have, they might require months or even years to revamp their whole application coding strategy while in the meantime, they would also be upgrading their existing servers and domain controllers to Windows Server 2008 R2. So let's first see what we are talking about here...

According to https://support.microsoft.com/kb/2009267 titled Windows Server 2008 R2 or Windows Server 2008 domain controller returns only 5000 attributes in a LDAP response:

“An LDAP application may return less information when a query is sent to a Windows Server 2008 or Windows Server 2008 R2 domain controller than when sent to a Windows Server 2003 domain controller. The query results may appear truncated or incomplete. In some occasions you may not get any results.

If, for example, a LDAP application queries the members of a group, the Windows Server 2008 R2 or Windows Server 2008 domain controller only returns 5000 members, while the Windows Server 2003 domain controllers returns many more members…."

and

"… Hardcoded LDAP limitations have been introduced in Windows Server 2008 R2 and Windows Server 2008 to prevent overloading the domain controller”

What this means is that Windows Server 2008 R2 or Windows Server 2008 dictates MaxPageSize of 20,000 and MaxValRange of 5,000 therefore the maximum number of attributes a query can return is 5,000.

CAUTION: The below should be tested first for any impact on performance as stated in the above KB, and it is also recommended to use Paged Queries (RFC 2969), a standard which was introduced ~10 years ago.

To override the upper-limits introduced in Windows Server 2008/R2 and restore the old-style (no upper limit enforced behavior for LDAP Query Policy in Windows Server 2003), modify the dSHeuristic attribute in Active Directory. To do this, follow these steps:

1. Start ADSI Edit. To do this, open a command prompt in the Support Tools folder, type ADSIEDIT.MSC, press Enter

2. Right-click CN=Directory Service in the following location, and then click Properties: CN=Directory Service,CN=Windows NT,CN=Services,CN=Configuration,DC=forest root

3. Click the Attribute Editor tab, and then locate dSHeuristic in the Attributes list.
Note By default, the value of this attribute is not set.

4. Click dSHeuristic, and then click Edit.

5. Type 000000000100000001 in the Value box, and then click OK. See Note below.

6. Restart the Active Directory Domain Service (NTDS) or the domain controller.

Note If a value has already been set for this attribute, incorporate the existing settings into the new value. When you do this, note the following:

·         The tenth character from the left must be 1. Twentieth bit must be 2, and so on.

·         The eighteenth character from the left must be 1.

·         None of the other characters of the existing value should be changed. For instance, if the existing value is 0000002 then the new value should be 000000200100000001

Microsoft Warning If you use the ADSI Edit snap-in, the LDP utility, or any other LDAP client, and you incorrectly modify the attributes of Active Directory objects, you can cause serious problems. These problems may require you to reinstall Microsoft Windows Server OS, Microsoft Exchange, or both Windows and Exchange. Microsoft cannot guarantee that problems that occur if you incorrectly modify Active Directory object attributes can be solved. Modify these attributes at your own risk. For more, please see below

Windows Server 2008 R2 or Windows Server 2008 domain controller returns only 5000 attributes in a LDAP response https://support.microsoft.com/kb/2009267

Change the LDAP Policy using NTDSUTIL, please follow https://support.microsoft.com/kb/315071

Please also see fLDAPBypassUpperBoundsOnLimits (2 bytes): dSHueristic Attribue

https://msdn.microsoft.com/enus/library/ms675656(VS.85).aspx

https://msdn.microsoft.com/en-us/library/cc223560(PROT.13).aspx

Comments

  • Anonymous
    January 01, 2003
    Excellent stuff - way to go!

  • Anonymous
    January 01, 2003
    Hi benguesmia, per msdn.microsoft.com/.../ms675656(VS.85).aspx DS-Heuristics is a Unicode string in which each character contains a value for a single domain-wide setting. The DS-Heuristics string takes the following format. | <1> | <2> | <3> | <4> | <5> | <6> | <7> | <8> | <9> | <10> | <11> | <12> | <13> | To provide data validation, each tenth character is set to the character number divided by ten. For example, the tenth character is '1'; the twentieth character is '2', and so on. HTH

  • Anonymous
    January 01, 2003
    wooow, cool blog Qasim.

  • Anonymous
    September 05, 2010
    wow, i did not think it was possible, yet, we were struggling to allow more than 5000 returns by a domain controller, you are absoultely right that it will take a lot of customers a long time to get to paged queries. In the meantime, its good to know there is a workaround we can use to breath. Thanks

  • Anonymous
    September 08, 2010
    Excellent Blog, thank you for sharing. I am sure we need to test our applications and OS requirements before migrating to Windows R2, this blog can help us buy more time.

  • Anonymous
    September 09, 2010
    good one indeed, we often don't see a blog like this from Microsoft. But I reckon, with workaround like this, why would companies force their coders to re-write their code or work according to the RFC query standard. Regardless, I am happy to see that there is a way, in the interim, atleast.

  • Anonymous
    November 30, 2010
    Yesssss, it worked we were blocked with our wifi router that authenticate users on Activedirectory (Fortigate) and the authentication module was unable to retrieve more than 5000 users in a group in spite of the fact that we had modified the ntds-settings. but in the microsoft article : msdn.microsoft.com/.../cc223560(PROT.13).aspx the bits start from 0 so th tenth bit is in fact the illeventh ! the bits you indicated starts from 1 so the tenth bit is the tenth : are they wrong ? i filled dsHeuristics with your version and it worked..

  • Anonymous
    January 31, 2011
    excellent, i was struggling with this and reading the comments plus your value, its working now. thanq again

  • Anonymous
    October 04, 2011
    good blog on how to defeat these limits, many thanks

  • Anonymous
    November 05, 2012
    Note sure if my long, detailed post was lost. To summarize, I think the info posted regarding MaxPageSize is incorrect. Reference: 3.1.1.3.4.6 LDAP Policies msdn.microsoft.com/.../cc223376(prot.20).aspx

  • Anonymous
    July 09, 2014
    Great article, but as suggested in the midst of the text page limits are in place for a reason. It allows sharing of the service with other applications. For large corporations with lots of objects, you can hang a DC by setting this to a higher value and letting an app query in large numbers. Something to keep in mind from a performance perspective. When programmers come to me with this complaint, I tell them to research paging thier query and they have a solution same day, no fuss. "Look my apps work!" is great, until users complain they cannot log on.

  • Anonymous
    December 11, 2014
    Thanks for sharing.