Using Domain Security: Configuring Mutual TLS

  • Article

 

Applies to: Exchange Server 2010 SP3, Exchange Server 2010 SP2

This topic explains how to configure mutual Transport Layer Security (TLS) for Domain Security, the set of functionality in Microsoft Exchange Server 2010 and Microsoft Office Outlook 2007 that provides a relatively low-cost alternative to S/MIME and other message-level security solutions.

For the purposes of this scenario, this topic explains how Exchange administrators at a fictitious company, Contoso, configure their Exchange 2010 environment to exchange domain-secured e-mail with their partner, Woodgrove Bank. Contoso administrators want to make sure that all e-mail sent to and received from Woodgrove Bank is protected with mutual TLS. Also, they want to configure Domain Security functionality so that all mail to and from Woodgrove Bank is rejected if mutual TLS can't be used.

Contoso has an internal public key infrastructure (PKI) that generates certificates. The PKI's root certificate has been signed by a major third-party certification authority (CA). Woodgrove Bank uses the same third-party CA to generate their certificates. Therefore, both Contoso and Woodgrove Bank trust the other's root CAs.

To set up mutual TLS, Exchange administrators at Contoso perform the following procedures:

Step 1: Generate a certificate request for TLS certificates

Step 2: Import certificates to Edge Transport servers

Step 3: Configure outbound Domain Security

Step 4: Configure inbound Domain Security

Step 5: Test domain-secured mail flow

Prerequisites

  • This topic assumes that you have read and understood Generate Request for Third-Party Certificate Services.

  • The Microsoft Exchange EdgeSync service must be fully deployed for Domain Security. Generally, configuration changes that are made to Domain Security functionality that don't use the ExchangeCertificate cmdlets are made within the organization and synchronized to Edge Transport servers by using the Microsoft Exchange EdgeSync service.

  • Before you can successfully run mutual TLS on an Edge Transport server, you must configure the computer and PKI environment so that certificate validation and certificate revocation list checking are operable. For more information, see Using PKI on the Edge Transport Server for Domain Security.

  • Even though individual configuration steps within this scenario can be accomplished with lesser rights, to complete the entire end-to-end scenario tasks, your account needs to be a member of the Organization Management management role group.

Step 1: Generate a Certificate Request for TLS Certificates

Contoso has an internal PKI that's subordinated to a third-party CA. In this scenario, subordination refers to the fact that the CA that's deployed by Contoso in the corporate infrastructure contains a root certificate that has been signed by a public third-party CA. By default, the public third-party CA is one of the trusted root certificates in the Microsoft Windows certificate store. Therefore, any client that includes the same third-party CA in its trusted root store and that connects to Contoso can authenticate to the certificate that's presented by Contoso.

Contoso has two Edge Transport servers that require TLS certificates: mail1.contoso.com and mail2.mail.contoso.com. Therefore, the Contoso e-mail administrator must generate two certificate requests, one certificate request for each server.

The following example shows the commands that the administrator uses to generate the base64-encoded PKCS#10 certificate requests.

The Contoso administrator must run this command for CN=mail1.contoso.com.

$Data1 = New-ExchangeCertificate -GenerateRequest -FriendlyName "Internet certificate for mail1" -SubjectName "DC=com,DC=Contoso,CN=mail1.contoso.com" -DomainName mail.contoso.com
Set-Content -Path "C:\Certificates\mail1-request.req" -Value $Data1

The Contoso administrator must run this command for CN=mail2.mail.contoso.com.

$Data2 = New-ExchangeCertificate -GenerateRequest -FriendlyName "Internet certificate for mail2" -SubjectName "DC=com,DC=Contoso,CN=mail2.mail.contoso.com"  -DomainName mail.contoso.com
Set-Content -Path "C:\Certificates\mail2-request.req" -Value $Data2

For detailed syntax and parameter information, see New-ExchangeCertificate.

Important

The specific details of the certificate or certificate request that you create depends on many variables. If you're generating a request, make sure that you work closely with the CA or the PKI administrator who will issue your certificate. For more information about how to create a certificate request for TLS, see Generate Request for Third-Party Certificate Services.

Return to top

Step 2: Import Certificates to Edge Transport Servers

After the Contoso administrator generates the certificate requests, the CA administrator for Contoso then uses the requests to generate the certificates for the servers. The resulting certificates must be issued as either a single certificate or a certificate chain and copied to the appropriate Edge Transport servers.

Important

Do not use the Certificate Manager snap-in in the Microsoft Management Console (MMC) to import the certificates for TLS on the Exchange server. Using the Certificate Manager snap-in to import certificates on Exchange servers doesn't bind the request that's created in this procedure to the issued certificate. Therefore TLS might fail. You can use the Certificate Manager snap-in to import certificates and keys that are stored as .pfx files into the local computer store.

When you import the certificate to the Edge Transport server, you must also enable the certificate for the SMTP service. The Contoso administrator runs the following command on each Edge Transport server, one time for each respective certificate.

Import-ExchangeCertificate -FileData ([Byte[]]$(Get-Content -Path C:\Certificates\mail1-certificate.pfx -Encoding Byte -ReadCount 0)) | Enable-ExchangeCertificate -Services SMTP

The preceding example imports and enables the TLS certificate by pipelining the certificate to the Enable-ExchangeCertificate cmdlet. You can also enable the certificate after you import it. If you do, you will need to specify the thumbprint of the certificate that you want to enable.

For detailed syntax and parameter information, see the Import-ExchangeCertificate and Enable-ExchangeCertificate topics.

When you receive a certificate from your PKI or CA provider, convert the issued certificate to a .pfx (PKCS#12) file so that you can back it up as part of a disaster contingency. A .pfx file contains the certificate and related keys. In some cases, you may want to transport the certificate and keys to move them to other computers. For example, if you have multiple Edge Transport servers where you expect to send and receive e-mail that's domain secured, you can create a single certificate that will work for all servers. In this case, you must have the certificate imported and enabled for TLS on each Edge Transport server.

As long as you keep a copy of the .pfx file securely archived, you can always import and enable the certificate. The .pfx file contains the private key so it's important to physically protect the file by keeping it on storage media in a secure location.

It's important to understand that the Import-ExchangeCertificate cmdlet always marks the imported private key from the .pfx file as non-exportable. This functionality is by design.

When you use the Certificate Manager snap-in in MMC to import a .pfx file, you can specify private key exportability and strong-key protection.

Important

Do not enable strong-key protection on certificates that are intended for TLS. Strong-key protection prompts the user every time that the private key is accessed. With Domain Security, the user is the SMTP service on the Edge Transport server.

Return to top

Step 3: Configure Outbound Domain Security

You must perform three steps to configure outbound Domain Security:

  1. Run the Set-TransportConfig cmdlet to specify the domain with which you want to send domain-secured e-mail.

  2. Run the Set-SendConnector cmdlet to set the DomainSecureEnabled property on the Send connector that will send mail to the domain with which you want to send domain-secured e-mail.

  3. Run the Get-SendConnector cmdlet to verify the following:

    • The Send connector that will send mail to the domain with which you want to send domain-secured e-mail routes mail with Domain Name System (DNS).

    • The FQDN is set to match either the Subject Name or the Subject Alternative Name of certificates that you're using for Domain Security.

Because the changes that you make in these three steps are global, you must make the changes on an internal Exchange server. The configuration changes that you make will be replicated to the Edge Transport servers by using the Microsoft Exchange EdgeSync service.

Step 3a: Specify the Sender Domain in Transport Configuration

It's relatively straightforward to specify the domain with which you want to send domain-secured e-mail. The Contoso administrator runs the following command on an internal Exchange 2010 server.

Set-TransportConfig -TLSSendDomainSecureList woodgrovebank.com

The TLSSendDomainSecureList parameter takes a multivalued list of domain names. The Set-TransportConfig command replaces the whole value for TLSSendDomainSecureList with the new value that's supplied in the cmdlet. Therefore, if you have other domains already configured and want to add a new domain, you need to either include the existing domain in the list or use a temporary variable. The following example shows how to add the woodgrovebank.com domain to the TLSSendDomainSecureList parameter without overwriting any existing values.

$TransportConfig = Get-TransportConfig
$TransportConfig.TLSSendDomainSecureList += "woodgrovebank.com"
Set-TransportConfig -TLSSendDomainSecureList $TransportConfig.TLSSendDomainSecureList

For detailed syntax and parameter information, see Set-TransportConfig.

Step 3b: Configure the Default Send Connector

Contoso will use the default DNS-routed Send connector named Internet to send domain-secured e-mail to their partners. Because their default DNS-routed Send connector is a default Internet Send connector, it uses DNS to route mail and doesn't use a smart host. The FQDN is already set to mail.contoso.com. Because the certificates that the Contoso administrator created set the DomainName parameter of the New-ExchangeCertificate to mail.contoso.com, the Send connector can use the certificates without additional configuration.

If you've configured a subdomain for testing, you may have to update the FQDN of your Send connector to match the certificate that you've created (for example, subdomain.mail.contoso.com). On the other hand, if you've created a certificate that contains the subdomain in the Subject or Subject Alternative name fields, you don't have to update the FQDN of the Send connector.

Therefore, the only configuration that the Contoso administrator must make to the Send connector is to set the DomainSecureEnabled parameter. To do this, the Contoso administrator runs the following command on an internal Exchange 2010 server for the Internet Send connector.

Set-SendConnector Internet -DomainSecureEnabled:$true

For detailed syntax and parameter information, see Set-SendConnector.

Step 3c: Verify the Send Connector Configuration

After completing the configuration changes, the Contoso administrator needs to verify that the Send connector being used for Domain Security is configured correctly. To do so, the Contoso administrator needs to run the following command.

Get-SendConnector Internet | Format-List Name,DNSRoutingEnabled,FQDN,DomainSecureEnabled

This command will list the relevant parameters configured for Domain Security, allowing the Contoso administrator to verify the configuration.

For detailed syntax and parameter information, see Get-SendConnector.

Return to top

Step 4: Configure Inbound Domain Security

You must perform two steps to enable inbound Domain Security:

  1. Run the Set-TransportConfig cmdlet to specify the domain from which you want to receive domain-secured e-mail.

  2. On the Edge Transport server, use the Exchange Management Shell or the Exchange Management Console (EMC) to enable Domain Security on the Receive connector from which you want to receive domain-secured e-mail. Because Domain Security requires mutual TLS authentication, TLS must also be enabled on the Receive connector.

Step 4a: Specify the Recipient Domain in Transport Configuration

It's relatively straightforward to specify the domain with which you want to receive domain-secured e-mail. To specify the domain, the Contoso administrator runs the following command in the Shell on an internal Exchange 2010 server or management workstation.

Set-TransportConfig -TLSReceiveDomainSecureList woodgrovebank.com

The TLSReceiveDomainSecureList parameter takes a multivalued list of domain names. The Set-TransportConfig command replaces the whole value for TLSReceiveDomainSecureList parameter with the new value supplied by the Set-TransportConfig cmdlet. Therefore, if you have other domains already configured and want to add a new domain, you need to either include the existing domain in the list or use a temporary variable. The following example shows how to add the woodgrovebank.com domain to the TLSReceiveDomainSecureList parameter without overwriting any existing values.

$TransportConfig = Get-TransportConfig
$TransportConfig.TLSReceiveDomainSecureList += "woodgrovebank.com"
Set-TransportConfig -TLSReceiveDomainSecureList $TransportConfig.TLSReceiveDomainSecureList

For detailed syntax and parameter information, see Set-TransportConfig.

Step 4b: Configure the Receive Connector

You must configure the Receive connector on each Edge Transport server that accepts mail from the domain from which you want to receive domain-secured e-mail. The Contoso environment is configured to have a single Internet Receive connector, with an Identity parameter value of Internet, on both Edge Transport servers. Therefore, to enable TLS while mail is sent to or received from Woodgrove Bank, the Contoso administrator must make sure that TLS is enabled on the default Internet Receive connector on both Edge Transport servers. To do so, the Contoso administrator runs the following command on both mail1.contoso.com and mail2.mail.contoso.com.

Set-ReceiveConnector Internet -DomainSecureEnabled $true -AuthMechanism TLS

For detailed syntax and parameter information, see Set-ReceiveConnector.

You can also use the EMC to configure the Receive connector using the following steps.

  1. On the Edge Transport server, open the EMC, click Edge Transport, and then in the result pane, click the Receive Connectors tab.

  2. Select the Receive connector that accepts mail from the domain from which you want to receive domain-secured e-mail, the connector Internet in this case, and then click Properties in the action pane.

  3. On the Authentication tab, select Transport Layer Security (TLS) and Enable Domain Security (Mutual Auth TLS), and then click OK.

Be aware that specifying the authentication mechanism as TLS doesn't force TLS on all inbound connections.

TLS will be forced for connections from Woodgrove Bank for the following reasons:

  • Woodgrove Bank is specified in the Set-TransportConfig cmdlet on the TLSReceiveDomainSecureList parameter.

  • The DomainSecureEnabled parameter is set to $true on the Receive connector.

Other senders that aren't listed on the TLSReceiveDomainSecureList parameter in the Set-TransportConfig cmdlet will only use TLS if TLS is supported by the sending system.

Return to top

Step 5: Testing Domain-Secured Mail Flow

After you've configured domain-secured e-mail, you can test the connection by reviewing the performance logs and the protocol logs. Messages that have successfully authenticated over the domain-secured mail flow path are displayed in Outlook as Domain Secure messages.

Performance Counters

The Domain Security feature includes the following set of performance counters under MSExchange Secure Mail Transport:

  • Domain-Secured Messages Received

  • Domain-Secured Messages Sent

  • Domain-Secured Outbound Session Failures

You can create a new counter log file for domain-secured mail flow with these performance counters to monitor the number of messages sent and received and also to monitor failed mutual TLS sessions. For more information about how to create and configure counter logs, see the Help file that's included with the Performance Logs and Alerts MMC snap-in.

Protocol Logs

You can review the send and receive protocol logs to determine whether TLS negotiation has been successful.

To view detailed protocol logs, you must set the protocol logging level to Verbose on the connectors that your organization uses to send and receive domain secured e-mail. To accomplish this, the Contoso administrator runs the following on both Edge Transport servers.

Set-ReceiveConnector Internet -ProtocolLoggingLevel Verbose

To enable protocol logging on the Send connector, the Contoso administrator runs the following on an internal Exchange server or management workstation. The configuration change is then replicated to the Edge Transport servers using the Microsoft Exchange EdgeSync service.

Set-SendConnector Internet -ProtocolLoggingLevel Verbose

For detailed syntax and parameter information, see the Set-ReceiveConnector and Set-SendConnector topics.

For more information about how to view protocol logs, see Configure Protocol Logging.

Return to top

 © 2010 Microsoft Corporation. All rights reserved.