Deploying a Basic Domain Isolation Policy

Updated: December 7, 2009

Applies To: Windows 7, Windows Server 2008, Windows Server 2008 R2, Windows Vista

By using Windows Firewall with Advanced Security, you can create connection security rules that specify that traffic must be secured by one or more of the features of IPsec. In domain isolation you use IPsec authentication to require each domain member computer involved in a connection to positively establish the identity of the other computer.

By creating rules that require authentication by a domain member, you effectively isolate those domain-member computers from computers that are not part of the domain.

Computers in a domain isolation environment require authentication for inbound connections. For outbound connections, you typically use the option to request but not require IPsec protection. This enables the computers to protect traffic when communicating with computers that can also use IPsec, but fall back to plaintext when communicating with computers that cannot use IPsec. With Windows XP and earlier versions of Windows, fall back to plain text, when it is enabled, occurs after trying IPsec for three seconds. However, some services have response time-outs that are less than three seconds, which causes them to fail. In these earlier versions of Windows this meant that you had to create (sometimes a very large number of) outbound exemption rules to support those servers or services that cannot authenticate. To address this problem, Microsoft released the Simple Policy Update for Windows Server 2003 and Windows XP. This update reduces the delay for attempts between IPsec-protected clients and non-IPsec-protected clients to one-half second. For more information about the Simple Policy Update for Windows Server 2003 and Windows XP, see Simplifying IPsec Policy with the Simple Policy Update at https://go.microsoft.com/fwlink/?LinkID=94767.

Later versions of Windows improve on this further, and no longer require the update. When you use request mode in Windows Vista and later versions of Windows, Windows sends both connection attempts at the same time. If the remote host responds with IPsec, then the non-IPsec attempt is abandoned. If the IPsec request generates no response then the non-IPsec attempt continues.

This reduced or eliminated delay solves the time-out failure problem for most programs. However, there might still be times when you want to make sure that your computers do not use IPsec to try to talk to certain hosts on the network. In those circumstances, create authentication exemption rules for the clients and they no longer use IPsec to communicate with computers on the exemption list.

For more information about domain isolation, see Introduction to Server and Domain Isolation (https://go.microsoft.com/fwlink/?LinkID=94631) and Domain Isolation with Microsoft Windows Explained (https://go.microsoft.com/fwlink/?LinkID=94632) in the Windows Server Technical Library.

Steps for creating connection security rules to enforce domain isolation

In this section, you create connection security rules that specify that the computers in your domain require authentication for inbound network traffic and request authentication for outbound traffic.

Step 1: Creating a Connection Security Rule that Requests Authentication

Step 2: Deploying and Testing Your Connection Security Rules

Step 3: Changing the Isolation Rule to Require Authentication

Step 4: Testing Isolation with a Computer That Does Not Have the Domain Isolation Rule

Step 5: Creating Exemption Rules for Computers that are Not Domain Members

Next topic: Step 1: Creating a Connection Security Rule that Requests Authentication