We have about 8 AD servers in total, 3 in HQ office, and the rest are in remote office, there is IPSEC vpn between offices to make the AD servers sync and replica with each other. PDC is in HQ office is running on Windows 2016 server.
Last week, we tried to patch all AD servers to be compliant with CVE-2020-1472. (https://portal.msrc.microsoft.com/en-US/security-guidance/advisory/CVE-2020-1472)
we managed get all AD servers patched, except one windows 2012 AD server in HQ office(Let me call it AD2), it looks like we are unable to install the security update on this server and in the end, the server is no long responding to client login/authentication neither.
So far we did not notice any other service has been impacted by the update, As I was trying to troubleshoot this AD2 server, I noticed a lot of errors and alert in the Directory service catalog, in the event viewer.
Log Name: Directory Service
Source: Microsoft-Windows-ActiveDirectory_DomainService
Date: 9/29/2020 10:55:12 PM
Event ID: 1789
Task Category: Knowledge Consistency Checker
Level: Error
Keywords: Classic
User: ANONYMOUS LOGON
Computer: AD2.domain.local
Description:
The site CN=Site1,CN=Sites,CN=Configuration,DC=domain,DC=local contains one or more directory servers, but is not connected by any site links. This site cannot replicate with other sites unless they are connected by site links.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">;
<System>
<Provider Name="Microsoft-Windows-ActiveDirectory_DomainService" Guid="{0e8478c5-3605-4e8c-8497-1e730c959516}" EventSourceName="NTDS KCC" />
<EventID Qualifiers="49152">1789</EventID>
<Version>0</Version>
<Level>2</Level>
<Task>1</Task>
<Opcode>0</Opcode>
<Keywords>0x8080000000000000</Keywords>
<TimeCreated SystemTime="2020-09-29T14:55:12.438350400Z" />
<EventRecordID>844600</EventRecordID>
<Correlation />
<Execution ProcessID="556" ThreadID="1868" />
<Channel>Directory Service</Channel>
<Computer> AD2.domain.local</Computer>
<Security UserID="S-1-5-7" />
</System>
<EventData>
<Data>CN=site1,CN=Sites,CN=Configuration,DC=domain,DC=local</Data>
</EventData>
</Event>
Log Name: Directory Service
Source: Microsoft-Windows-ActiveDirectory_DomainService
Date: 9/29/2020 10:55:12 PM
Event ID: 1311
Task Category: Knowledge Consistency Checker
Level: Error
Keywords: Classic
User: ANONYMOUS LOGON
Computer: AD2.domain.local
Description:
The Knowledge Consistency Checker (KCC) has detected problems with the following directory partition.
Directory partition:
DC=domain,DC=local
There is insufficient site connectivity information for the KCC to create a spanning tree replication topology. Or, one or more directory servers with this directory partition are unable to replicate the directory partition information. This is probably due to inaccessible directory servers.
User Action
Perform one of the following actions:
- Publish sufficient site connectivity information so that the KCC can determine a route by which this directory partition can reach this site. This is the preferred option.
- Add a Connection object to a directory service that contains the directory partition in this site from a directory service that contains the same directory partition in another site.
If neither of the tasks correct this condition, see previous events logged by the KCC that identify the inaccessible directory servers.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">;
<System>
<Provider Name="Microsoft-Windows-ActiveDirectory_DomainService" Guid="{0e8478c5-3605-4e8c-8497-1e730c959516}" EventSourceName="NTDS KCC" />
<EventID Qualifiers="49152">1311</EventID>
<Version>0</Version>
<Level>2</Level>
<Task>1</Task>
<Opcode>0</Opcode>
<Keywords>0x8080000000000000</Keywords>
<TimeCreated SystemTime="2020-09-29T14:55:12.438350400Z" />
<EventRecordID>844606</EventRecordID>
<Correlation />
<Execution ProcessID="556" ThreadID="1868" />
<Channel>Directory Service</Channel>
<Computer>AD2.domain.local</Computer>
<Security UserID="S-1-5-7" />
</System>
<EventData>
<Data>DC=domain,DC=local</Data>
</EventData>
</Event>
Looks I have above messages on all AD servers and, and seems like the inter-site transports is not working, all site links are not working neither, none of the AD is able to talk to each, ran repadmin/replsum, 0 failures and there is no error returned.
I ran repadmin /syncall on the PDC:
repadmin /syncall CALLBACK MESSAGE: The following replication is in progress:
From: 1f0d3955-af2a-4161-a889-9276aa5ffdbc._msdcs.domain.local
To : 299cafa1-4f87-4a31-98b1-73dcba7abb86._msdcs.domain.local
CALLBACK MESSAGE: The following replication completed successfully:
From: 1f0d3955-af2a-4161-a889-9276aa5ffdbc._msdcs.domain.local To : 299cafa1-4f87-4a31-98b1-73dcba7abb86._msdcs.domain.local
CALLBACK MESSAGE: The following replication is in progress:
From: d6e77008-6514-4dad-af22-f558d8003879._msdcs.domain.local
To : 299cafa1-4f87-4a31-98b1-73dcba7abb86._msdcs.domain.local
CALLBACK MESSAGE: The following replication completed successfully: From: d6e77008-6514-4dad-af22-f558d8003879._msdcs.domain.local
To : 299cafa1-4f87-4a31-98b1-73dcba7abb86._msdcs.domain.local
CALLBACK MESSAGE: SyncAll Finished.
SyncAll terminated with no errors.
much appreciate your help on this