Active Directory: Allow Linked Multi-Valued Attributes to use LVR
If values were added to linked multi-valued attributes before the Forest Functional Level (FFL) was raised to at least Windows Server 2003, the attribute will not be able to take advantage of linked value replication (LVR). These are called "Legacy" values. Without LVR, the replication of these attributes will be much less efficient.
Introduction
In Windows 2000 Active Directory, when a linked multi-valued attribute was updated, the entire attribute had to replicate to all domain controllers. Starting with Windows Server 2003 forest functional level (FFL), such attributes could use linked value replication (LVR). With LVR, only the new or updated values need to replicate. But any values added before the FFL was upgraded to Windows Server 2003 will not take advantage of LVR. The result will be greatly increased replication traffic when the attributes are updated. This article describes how to remedy this situation.
Linked Multi-Valued Attributes
Multi-valued attributes in Active Directory are similar to arrays. They can have zero, one, or many separate values. Examples include the member attribute of groups and the memberOf, proxyAddresses, businessCategory, and otherTelephone attributes of users.
Linked attributes are pairs of attributes. The forward link is an attribute you can update. The back link is a related attribute that is automatically updated by the system when you update the forward link. For example, the member attribute of groups is a forward link. When you add a user to a group, the distinguished name of the user is added to the member attribute. Then the system automatically adds the distinguished name of the group to the memberOf attribute of the corresponding user.
Only the forward linked attribute is actually saved in the Active Directory database. A link table is used to link the back linked attribute. For example, the member attribute of a group is an array of member distinguished names saved with the group object in AD. But the memberOf attribute of a user simply uses the link table to refer to the distinguished name of the group.
Replication of Linked Multi-Valued Attributes with FFL Windows 2000
In Windows 2000 Active Directory, if a group had 1,000 members, and you added another member, all 1,001 members had to replicate to all other domain controllers in the domain. This could result in a lot of replication traffic. As a result, groups in Windows 2000 were restricted to about 5,000 direct members.
Membership in the "primary group" of a user, usually "Domain Users", is not saved in the member and memberOf attributes. This group most likely includes all users in the domain. Instead, membership in the "primary group" is saved in the primaryGroupID attribute of the user (or computer). This numeric value matches the value of the primaryGroupToken attribute of the group. This allowed the "Domain Users" group to include far more than 5,000 members, without generating large replication traffic when a new user was added.
Replication of Linked Multi-Valued Attributes with FFL Windows Server 2003
Starting with Windows Server 2003 FFL, linked multi-valued attributes take advantage of linked value replication (LVR). With LVR, when a new member is added to a group with 1,000 existing members, only the new distinguished name must replicate to the other domain controllers. This only applies to the forward linked attribute. The back linked attribute, such as the memberOf attribute, is linked through the link table in Active Directory. LVR only applies to cases where the forward linked attribute is multi-valued. For example, the manager attribute is linked to the directReports attribute, but the forward linked attribute in this pair, the manager attribute, is single valued. The manager attribute does not need to use LVR to reduce replication traffic.
However, when the forest is upgraded from Windows 2000 FFL to Windows Server 2003 or above, any existing values of the linked multi-valued attributes cannot take advantage of LVR. These values are called "legacy". If your Active Directory was created years ago, you may have many object attributes that are not using linked value replication. Most such cases will be the member attribute of groups populated with many members in the past. However, other forward linked multi-valued attributes could also have this problem. Examples include msDS-HasMasterNCs, msDS-NC-Replica-Locations, msDS-HostServiceAccount, and siteList. Some of these attributes were introduced with newer versions of Active Directory, like Windows Server 2008. But just because the schema has been updated for the new attributes, if the Forest Functional Level has not been raised above Windows 2000, any values added to these attributes will not take advantage of LVR.
The TechNet library article linked in the "Other Resources" section below estimates the replication traffic resulting from each entry in the member attribute of groups. The estimate is about 180 bytes per group member. If a group has 1,000 "legacy" values in the member attribute, and one more is added, this would result in 180,000 extra bytes of replication traffic.
Script to Find Attributes With Possible Legacy Values
A PowerShell script has been developed to find all attributes of all objects in Active Directory that have forward linked multi-valued attributes with at least a minimum number of values. All naming contexts are searched, including the Schema and Configuration naming contexts. The script outputs the distinguished names of the objects, the lDAPDisplayName of the attribute, and the number of values. If any of these values were added before the FFL was raised to at least Windows Server 2003, they could be legacy. If they are legacy, they will not take advantage of linked value replication. The script is linked here.
The variable $Limit in the script specifies the minimum number of values in each attribute before the script outputs. You can modify this limit to any value desired. If the limit is 1, the script will output all forward linked multi-valued attributes of all objects with any values.
How to Verify Legacy Values
The script FindLegacy.ps1 identifies object attributes that might have legacy values. Any object attributes with many values would warrant further investigation. The most likely candidates are the member attributes of groups, but other attributes might also deserve further attention.
The repadmin command line utility can be used to verify which values are legacy. The utility would be run separately for each object in the output from FindLegacy.ps1. The repadmin command outputs information for all forward linked multi-valued attributes of each object. The command would be similar to.
repadmin /showobjmeta mydc "cn=My Object,ou=West,dc=MyDomain,dc=com" > report.txt
where "mydc" is the host name of a domain controller and the distinguished name is that of any object in Active Directory. The output should be redirected to a text file. Near the end of the output, all values of any linked multi-valued attributes will be listed. If the "type" is "PRESENT", that means the value was added after the FFL was raised to Windows Server 2003 or above. Such values will include metadata. If the "type" is "ABSENT", that means the value was removed, but the tombstone lifetime has not yet expired. If the "type" is "LEGACY", that means the value was added when the FFL was still Windows 2000. These are the values that need to be fixed to take advantage of linked value replication.
Example output from the repadmin command follows.
Type Attribute Last Mod Time Originating DSA Loc.USN Org.USN Ver
======= ============ ============= ================= ======= ======= ===
Distinguished Name
=============================
LEGACY member
CN=School,OU=West,DC=MyDomain,DC=com
LEGACY member
CN=Adams\, John,OU=Students,OU=West,DC=MyDomain,DC=com
LEGACY member
CN=Washington\, George,OU=Students,OU=West,DC=MyDomain,DC=com
LEGACY member
CN=James M. Polk,OU=Test,OU=West,DC=MyDomain,DC=com
PRESENT member 2013-01-02 12:14:18 MyDomain-Lab\MYDC03 4900099 4900099 1
CN=Michele Geiger,OU=Students,OU=West,DC=MyDomain,DC=com
LEGACY member
CN=Roy Rogers,OU=Staff,OU=West,DC=MyDomain,DC=com
ABSENT member 2013-03-15 15:18:49 MyDomain-Lab\MYDC03 4900113 4900113 1
CN=Isaac Newton,OU=Students,OU=West,DC=MyDomain,DC=com
LEGACY member
CN=Sam Wilton,OU=Test,OU=West,DC=MyDomain,DC=com
Notice that metadata is only shown for values marked "PRESENT" or "ABSENT".
How to Fix Legacy Values
The fix for legacy values of these attributes is to first remove them, and then add them back. This would be worthwhile if there are more than a few values, especially if there are hundreds or thousands. Doing this manually, however, is not reasonable. Fortunately, there are several scripting options available.
Windows Server 2012 Script to Fix Legacy Members of a Group
The first link in the "Other Resources" section below is a blog post that includes a script to remediate legacy members of groups. This script uses a new cmdlet introduced in Windows Server 2012. The new cmdlet is Get-ADReplicationAttributeMetadata. The blog post shows how to use the cmdlet to retrieve an array of legacy members for one group. The Remove-ADGroupMember and Add-ADGroupMember cmdlets are used to first remove these members, and then add them back into the group.
PowerShell Version 2 Script to Fix Legacy Members of a Group
A PowerShell Version 2 script has been developed to do the same thing, using the output from the repadmin tool to identify the legacy members. The script is linked here.
This script also uses the Remove-ADGroupMember and Add-ADGroupMember cmdlets. However, it parses the output from the repadmin command to identify the legacy members of the group. This script requires the Active Directory PowerShell module. Be sure to modify the value assigned to $Server in the script. It should be the DNS name of a domain controller in your domain. This ensures that all removes and adds are done on the same domain controller. The script prompts for either the sAMAccountName or the distinguishedName of the group and the text file containing the output from the repadmin command run on the same group. The legacy members are processed in blocks of 4000 at most, to avoid excessive network traffic and long-running transactions.
PowerShell Version 2 Script to Fix Legacy Values of any Object Attribute
Another PowerShell Version 2 script was developed to fix the legacy values of any attribute. It remediates any forward linked multi-valued attribute of any object. Be sure to modify the value assigned to $Server in the script. It should be the DNS name of a domain controller in your domain. This ensures that all removes and adds are done on the same domain controller. The script is linked here.
This script also parses the output from the repadmin command to identify the legacy values of any specified attribute. It uses the -Remove and -Add parameters of the Set-ADObject cmdlet to remove the values and then add them back. The script prompts for either the sAMAccountName or the distinguishedName of an object, the lDAPDisplayName of the attribute, and the text file containing the output from the repadmin command run on the same object. The legacy values are processed in blocks of 4000 at most, to avoid excessive network traffic and long-running transactions.
After FixLegacyValues.ps1 is used on the same object as the one documented above, output from the repadmin command shows the legacy values have been fixed.
Type Attribute Last Mod Time Originating DSA Loc.USN Org.USN Ver
======= ============ ============= ================= ======= ======= ===
Distinguished Name
=============================
PRESENT member 2015-09-02 17:38:48 MyDomain-Lab\MYDC03 4900521 4900521 2
CN=School,OU=West,DC=MyDomain,DC=com
PRESENT member 2015-09-02 17:38:48 MyDomain-Lab\MYDC03 4900522 4900522 2
CN=Adams\, John,OU=Students,OU=West,DC=MyDomain,DC=com
PRESENT member 2015-09-02 17:38:48 MyDomain-Lab\MYDC03 4900523 4900523 2
CN=Washington\, George,OU=Students,OU=West,DC=MyDomain,DC=com
PRESENT member 2015-09-02 17:38:48 MyDomain-Lab\MYDC03 4900524 4900524 2
CN=James M. Polk,OU=Test,OU=West,DC=MyDomain,DC=com
PRESENT member 2013-01-02 12:14:18 MyDomain-Lab\MYDC03 4900099 4900099 1
CN=Michele Geiger,OU=Students,OU=West,DC=MyDomain,DC=com
PRESENT member 2015-09-02 17:38:48 MyDomain-Lab\MYDC03 4900525 4900525 2
CN=Roy Rogers,OU=Staff,OU=West,DC=MyDomain,DC=com
ABSENT member 2013-03-15 15:18:49 MyDomain-Lab\MYDC03 4900113 4900113 1
CN=Isaac Newton,OU=Students,OU=West,DC=MyDomain,DC=com
PRESENT member 2015-09-02 17:38:48 MyDomain-Lab\MYDC03 4900526 4900526 2
CN=Sam Wilton,OU=Test,OU=West,DC=MyDomain,DC=com
Metadata is now shown for all values. The values previously marked "LEGACY" are marked "PRESENT". The attribute will now take advantage of linked value replication.
See Also
- Wiki: Active Directory Domain Services (AD DS) Portal
- Wiki: Portal of TechNet Wiki Portals
- Active Directory: Glossary
- Active Directory: Non-Linked Multi-Valued Attribute Size Limits
- How Active Directory Replication Works
Other Resources
- Remediate Active Directory Members that Don't Support LVR (Hey, Scripting Guy! blog post by Ian Farr [MSFT])
- Find All AD Objects With Forward Linked Multi-Valued Attribute Values (PowerShell script FindLegacy.ps1)
- Fix Legacy Members of a Specified AD Group (PowerShell script FixLegacyMembers.ps1)
- Fix Legacy Values of a Specified AD Object Attribute (PowerShell script FixLegacyValues.ps1)
- Active Directory Replication Traffic
- Repadmin
- How the Active Directory Replication Model Works
- How the Data Store Works