Understanding Unified Messaging VoIP Security

Applies to: Exchange Server 2010 SP3, Exchange Server 2010 SP2

An important aspect of your network security is the ability to protect your Unified Messaging (UM) infrastructure. There are components within your Unified Messaging environment that you must correctly configure to help protect the data sent and received from Unified Messaging servers on your network. These include components such as Unified Messaging servers and dial plans. This topic discusses how you can increase protection for the Unified Messaging network data and servers in your organization. You must follow these steps to help secure your Unified Messaging environment and enable Voice over IP (VoIP) security:

1. Install the Unified Messaging server role.

2. Create a new self signed or public certificate that you can use for mutual TLS.

3. Associate a certificate with the UM server.

4. Configure the UM dial plan as SIP Secured or Secured.

5. Configure the startup mode on the UM server.

6. Associate the Unified Messaging servers with the UM dial plan.

7. Configure the UM IP gateways used to have a fully qualified domain name (FQDN) and to use TCP port 5061.

8. Export and import the required certificates to enable the Unified Messaging servers, IP gateways, IP Private Branch eXchanges (IP PBXs), and other servers running Microsoft Exchange Server 2010 to use mutual Transport Layer Security (mutual TLS).

Contents

Protecting Unified Messaging

Types of Certificates

Configuring Mutual TLS

IPsec

UM Dial Plans and VoIP Security

How Unified Messaging Determines Security Mode and Selects Certificates

Protecting Unified Messaging

There are several security methods that can help you protect your Unified Messaging servers and the network traffic sent between your IP gateways and Unified Messaging servers and between your Unified Messaging servers and other Exchange 2010 servers in your organization. The following table lists some possible threats to your Unified Messaging infrastructure and the security methods that can be implemented to help protect it.

Protecting Unified Messaging

There are several security methods listed in the previous table that you can use to protect your Unified Messaging environment. One of the most important mechanisms for protecting your Unified Messaging infrastructure and the network traffic generated by Unified Messaging is mutual TLS.

You can use mutual TLS to encrypt Voice over IP (VoIP) traffic passed between IP gateways, IP PBXs, and other Exchange 2010 servers and the Unified Messaging servers on your network. The best choice for protecting this data is to use mutual TLS to encrypt the VoIP data.

However, depending on the security threat, you can also configure IPsec policies to enable data encryption between IP gateways or IP PBXs and a Unified Messaging server or between a Unified Messaging server and other Exchange 2010 servers on your network. In some environments, you might be unable to use IPsec because IPsec may be unavailable or may not be supported on the IP gateways or IP PBXs. Additionally, IPsec puts an additional processing load on system resources on Unified Messaging servers. Considering these two factors, mutual TLS is a better choice for protecting the VoIP network traffic in your Unified Messaging environment.

After you correctly implement and configure mutual TLS, the VoIP traffic between the IP gateways, IP PBXs, and from other Exchange servers to the Unified Messaging servers is encrypted. However, when mutual TLS cannot be used to help secure the traffic sent or received from a Unified Messaging server, such as when a Unified Messaging server communicates with another server on your network, such as an Active Directory domain controller or an Exchange 2010 Mailbox server, other types of encryption are used to protect the data. The following figure shows the methods of encryption that you can use to protect Unified Messaging.

UM VoIP security

Return to top

Types of Certificates

Digital certificates are electronic files that work like an online passport to verify the identity of a user or computer and are used to create an encrypted channel to protect data. A certificate is basically a digital statement issued by a certification authority (CA) that vouches for the identity of the certificate holder and enables the parties to communicate in a secure manner using encryption. They can be issued by a trusted third-party CA, such as Certificate Services, or be self-signed. Each type of certificate has its advantages and disadvantages. However, certificates are always tamperproof and cannot be forged. Certificates can be issued for a variety of functions, such as Web user authentication, Web server authentication, S/MIME, IPsec, TLS, and code signing.

A certificate binds a public key to the identity of the person, computer, or service that holds the corresponding private key. The public and private keys are used by both the client and the server to encrypt the data before it's transmitted across the wire. Certificates are used by a variety of public key security services and applications that provide authentication, data integrity, and secure communications across networks such as the Internet. For Windows-based users, computers, and services, trust in a CA is established when there's a copy of the root certificate in the trusted root store and the certificate contains a valid certification path. This means that no certificates in the certification path have been revoked or have had the validity period expire.

Digital certificates do the following:

• They authenticate that their holders—people, Web sites, and even network resources such as routers—are truly who or what they claim to be.

• They protect data exchanged online from theft or tampering.

There are traditionally three options or kinds of certificates that Unified Messaging and IP gateways or IP PBXs can use. In all three approaches or options, the public key of the certificate owner is part of the certificate so that the server, user, Web site, or other resource on the other end can decrypt the messages. The private key is known only to the signer of the certificate. Each certificate has an EnhancedKeyUsage attribute set on it to dictate the specific usage for the certificate. For example, usage could be specified only for server authentication or for use with the encrypting file system. Unified Messaging uses the certificate for server authentication and data encryption.

Self-Signed Certificates

A self-signed certificate is a certificate signed by its own creator. The subject and the name of the certificate match. On self-signed certificates, the issuer and subject are defined on the certificate. Self-signed certificates don't require the presence of a CA from your organization or from a third party. You must configure these certificates explicitly and copy them to the trusted root certificate store on each IP gateway, IP PBX, other Unified Messaging servers, and other Exchange 2010 computers if they're to be trusted by the Unified Messaging server that has issued the certificate.

If a public key infrastructure (PKI)-based or third-party certificate is unavailable, the Unified Messaging server will search for a self-signed certificate in the local certificate store. If it cannot find a PKI or third-party certificate, it will generate a self-signed certificate for mutual TLS. However, because it's a self-signed certificate, it won't be trusted by the IP gateways, IP PBXs on the network, or other servers on the network. To make sure that the self-signed certificate is trusted by IP gateways, IP PBXs, or other servers, you must import the self-signed certificate into the local trusted root certificate store for the devices and servers. After you do this, when the Unified Messaging server presents this self-signed certificate to the IP gateway, IP PBX, or server, it will be able to verify that the certificate was issued by a trusted authority because the issuer will equal the subject defined on the self-signed certificate.

If you're using only self-signed certificates, you must import a single self-signed certificate for each IP gateway, IP PBX, or server. In large network environments that have multiple devices or computers, this may not be the best choice for implementing mutual TLS. Using self-signed certificates in a large enterprise network doesn't scale well because of the additional administrative overhead. However, administrative overhead isn't a problem if you have multiple devices and you're using a PKI or commercial third-party certificate. This is because each device has a certificate that has been issued by the same trusted root authority. Having a certificate from the same trusted root authority guarantees that all IP gateways, IP PBXs, and other servers trust the Unified Messaging server.

For mutual TLS to work using self-signed certificates:

1. Take the Unified Messaging server's self-signed certificate and import it into the trusted root certificate store on each IP gateway and IP PBX and on other servers that the Unified Messaging server will communicate with using mutual TLS.

2. Take the self-signed certificate from each IP gateway, IP PBX, and other server and import it into the Unified Messing server's trusted root certificate store. If you're using a PKI or third-party certificate, you will import the certification authority's certificate into the trusted root certificate store on all devices and servers.

Self-signed certificates are frequently not the best certificate option when you deploy mutual TLS or certificate-based authentication. However, smaller organizations with a limited number of devices or computers may decide to use the self-signed certificate method because it's the most easy to configure and the least expensive method to use when you implement mutual TLS. Frequently, smaller organizations decide not to use a third-party certificate or to install their own PKI to issue their own certificates because of the expense, because their administrators lack the experience and knowledge to create their own certificate hierarchy, or for both reasons. The cost is minimal and the setup is simple when you're using self-signed certificates. However, establishing an infrastructure for certificate life-cycle management, renewal, trust management, and revocation is much more difficult with self-signed certificates. For more information about how to create a certificate for TLS, see Understanding TLS Certificates.

Return to top

Public Key Infrastructure

A PKI is a system of digital certificates, CAs, and registration authorities (RAs) that verify and authenticate the validity of each party involved in an electronic transaction using public key cryptography. When you implement a CA in an organization that uses Active Directory, you provide an infrastructure for certificate life-cycle management, renewal, trust management, and revocation. These qualities provide a solid infrastructure for all the certificates in your organization. However, there's some cost involved in deploying additional servers and infrastructure to create and manage these types of certificates.

You can install Certificate Services on any server in the domain. If you obtain certificates from a domain Windows-based CA, you can use the CA to request or sign certificates to issue to your own servers or computers on your network. This enables you to use a PKI that resembles using a third-party certificate vendor but is less expensive. Although these PKIs cannot be deployed publicly, as other types of certificates can be, when a PKI is used, a CA signs the requestor’s certificate using the private key, and the requestor is verified. The public key of this CA is included with the certificate issued by the CA. Anyone who has this CAs certificate as a root certificate can use that public key to decrypt the requestor’s certificate and authenticate the requestor.

When you use a PKI certificate to implement mutual TLS, you must copy the required certificates to the IP gateways or IP PBXs. Then you must copy the certificates on the IP gateways or IP PBXs to the Unified Messaging servers associated with the UM dial plan that has been configured in secured mode.

The setup and configuration for using PKI certificates and third-party certificates resemble the procedures that you perform when importing and exporting the self-signed certificates. However, you mustn't only install the computer certificate into the trusted root certificate store. You must also import or copy the trusted root certificate for the PKI into the trusted root certificate store on the Unified Messaging servers and the IP gateways and IP PBXs on your network.

To deploy mutual TLS when you've already deployed a PKI infrastructure, follow these steps:

1. Generate a certificate request on each IP gateway or PBX.

2. Copy the certificate request to use when requesting the certificate from a certification authority.

3. Request a certificate from the certification authority using the certificate request. Save the certificate.

4. Import the certificate that you saved onto each device or computer.

5. Download the trusted root certificate for your PKI.

6. Import the trusted root certificate from your PKI on each device. If you're importing the trusted root certificate on an Exchange 2010 computer running the Unified Messaging server role, you can also use Group Policy to import the trusted root certificate into the trusted root certificate store on the Unified Messaging server or other Exchange 2010 servers. However, this process is also used when you're configuring a server running the Unified Messaging server role.

   Note

   You will use the same steps if you're using a commercial third-party certificate to implement mutual TLS.

For more information about certificates and PKIs, see the following topics:

• For more information about certificates, see Public Key Infrastructure for Windows Server 2003.

• For more information about best practices for implementing a Windows Server 2003 public key infrastructure, see Best Practices for Implementing a Microsoft Windows Server 2003 Public Key Infrastructure.

• For more information about how to deploy a Windows Server 2008-based PKI, see the Step-By-Step Example Deployment of the PKI Certificates Required for Configuration Manager Native Mode: Windows Server 2008 Certification Authority.

• For more information about Windows Server 2008 Certificate Services for a Windows Server 2008-based PKI, see Active Directory Certificate Services Overview.

Third-Party Certification Authorities

Third-party or commercial certificates are certificates generated by a third-party or commercial CA and then purchased for you to use on your network servers. One problem with self-signed and PKI-based certificates is that, because the certificate isn't trusted, you must make sure that you import the certificate into the trusted root certificate store on client computers, servers, and other devices. Third-party or commercial certificates don't have this problem. Most commercial CA certificates are already trusted because the certificate already resides in the trusted root certificate store. Because the issuer is trusted, the certificate is also trusted. Using third-party certificates greatly simplifies deployment.

For larger organizations or organizations that must publicly deploy certificates, it's best to use a third-party or commercial certificate, even though there's a cost associated with the certificate. Commercial certificates may not be the best solution for smaller and medium-size organizations, and you might decide to use one of the other certificate options available.

Depending on the configuration of the IP gateway or IP PBX, you might still have to import the third-party or commercial certificate into the trusted certificate store on the IP gateways and IP PBXs to be able to use the third-party certificate for mutual TLS. However, in some cases, the third-party certificate will be included in the trusted root certificate store on your Unified Messaging server and other Exchange 2010 computers in your organization.

The procedures that you perform to use a commercial third-party certificate for enabling mutual TLS are the same procedures that you perform when you use a PKI certificate. The only difference is that you won't have to generate a PKI certificate because you've purchased a certificate from a commercial third-party certificate vendor that will be imported into the trusted root certificate store on the servers and devices on your network.

Return to top

Configuring Mutual TLS

By default, when an incoming call is received from an IP gateway, the VoIP traffic isn't encrypted and doesn't use mutual TLS. However, the security setting for a Unified Messaging server is configured on the UM dial plan associated with the Unified Messaging server. To enable the Unified Messaging server to communicate securely with IP gateways, IP PBXs, and other Exchange 2010 servers, you must use the Set-UMDialPlan cmdlet to configure VoIP security on the UM dial plan, and then enable mutual TLS for the Unified Messaging servers associated with the UM dial plan.

After you've enabled VoIP security on the UM dial plan, all Unified Messaging servers associated with the UM dial plan can communicate in a secure manner. However, depending on the type of certificate that you use for enabling mutual TLS, you must first import and export the required certificates both on the Unified Messaging servers and the IP gateways and PBXs. After the required certificate or certificates have been imported on the Unified Messaging server, you must restart the Microsoft Exchange Unified Messaging service to be able to use the certificate that was imported to establish an encrypted connection with the IP gateways or IP PBXs. For more information about how to import and export certificates, see Import and Export Certificates.

After you successfully import and export the required trusted certificates, the IP gateway will request a certificate from the Unified Messaging server, and then it will request a certificate from the IP gateway. Exchanging the trusted certificates between the IP gateway and the Unified Messaging server enables the IP gateway and Unified Messaging server to communicate over an encrypted connection using mutual TLS. When an incoming call is received by an IP gateway or IP PBX, it will initiate a certificate exchange and negotiate security using mutual TLS with the Unified Messaging server. The Microsoft Exchange Unified Messaging service isn't involved in the certificate exchange process or in determining whether the certificate is valid. However, if a trusted certificate cannot be located on a Unified Messaging server, a trusted certificate is found but isn't valid, or a call is rejected because of a mutual TLS negotiation failure, the Unified Messaging server will receive a notification from the Microsoft Exchange Unified Messaging service.

Although the Microsoft Exchange Unified Messaging service doesn't participate in the certificate exchange between the Unified Messaging server and the IP gateways, the Microsoft Exchange Unified Messaging service does the following:

• Provides a list of FQDNs to the Microsoft Exchange Speech service so that calls from only the IP gateways or IP PBXs included on the list are accepted.

• Passes the issuerName and SerialNumber attributes of a certificate to the Microsoft Exchange Speech service. These attributes uniquely identify the certificate that the Unified Messaging server will use when an IP gateway or IP PBX requests a certificate.

After the Unified Messaging server and the IP gateways or IP PBXs have performed the key exchange to establish an encrypted connection using mutual TLS, the Unified Messaging servers will communicate with the IP gateways and IP PBXs using an encrypted connection. The Unified Messaging servers will also communicate with other Exchange 2010 servers, such as Client Access servers and Hub Transport servers, using an encrypted connection that uses mutual TLS. However, mutual TLS will only be used to encrypt the traffic or messages submitted from the Unified Messaging server to a Hub Transport server.

Return to top

IPsec

IPsec also uses certificates to encrypt data. It provides a key line of defense against private network and Internet attacks.

IPsec has the following goals:

• To protect the contents of IP packets.

• To defend against network attacks through packet filtering and the enforcement of trusted communication.

IPsec is a framework of open standards that helps ensure private, secure communications over IP networks using cryptographic security services.

IPsec uses cryptography-based protection services, security protocols, and dynamic key management. It provides the strength and flexibility to protect communications between private network computers, domains, sites, remote sites, extranets, and dial-up clients. It can even be used to block receipt or transmission of specific types of traffic.

IPsec is based on an end-to-end security model that establishes trust and security from a source IP address to a destination IP address. The IP address itself doesn't have to be considered an identity. Instead, the system behind the IP address has an identity validated through an authentication process. The only computers that must know about the traffic being secured are the sending and receiving computers. Each computer handles security at its respective end and operates under the assumption that the medium over which the communication occurs isn't secure. Computers that route data only from source to destination aren't required to support IPsec unless firewall-type packet filtering or network address translation is being done between the two computers. This enables IPsec to be deployed successfully for the following organizational scenarios:

• LAN   Client-to-server, server-to-server, and server-to-VoIP device

• WAN   Router-to-router and gateway-to-gateway

• Remote access   Dial-up clients and Internet access from private networks

Typically, both sides require IPsec configuration to set options and security settings that allow two systems to agree on how to help secure traffic between them. This is known as an IPsec policy. The Microsoft Windows 2000 Server, Windows XP, Windows Server 2003, and the Windows Server 2008 operating system implementations of IPsec are based on industry standards that were developed by the Internet Engineering Task Force (IETF) IPsec working group. Parts of IPsec-related services were jointly developed by Microsoft and Cisco Systems, Inc. For more information about how to configure IPsec policies, see Creating, modifying, and assigning IPSec policies.

For more information about IPsec, see IPSec Concepts.

UM Dial Plans and VoIP Security

Unified Messaging servers can communicate with IP gateways, IP PBXs, and other Exchange 2010 computers in either unsecured, SIP secured, or secured mode depending on how the UM dial plan is configured. A Unified Messaging server can operate in any mode configured on a dial plan because the Unified Messaging server is configured to listen on TCP port 5060 for unsecured requests and TCP port 5061 for secured requests at the same time. A Unified Messaging server can be associated with a single or multiple UM dial plans and can be associated with dial plans that have different VoIP security settings. A single Unified Messaging server can be associated with dial plans configured to use a combination of unsecured, SIP secured, or secured mode.

By default, when you create a UM dial plan, it will communicate in an unsecured mode, and the Unified Messaging servers associated with the UM dial plan will send and receive data from IP gateways, IP PBXs, and other Exchange 2010 computers using no encryption. In unsecured mode, both the Realtime Transport Protocol (RTP) media channel and SIP signaling information won't be encrypted.

You can configure a Unified Messaging server to use mutual TLS to encrypt the SIP and RTP traffic sent and received from other devices and servers. When you add a Unified Messaging server to a UM dial plan and configure the dial plan to use SIP secured mode, only the SIP signaling traffic will be encrypted, and the RTP media channels will still use TCP. TCP isn't encrypted. However, if you add a Unified Messaging server to a UM dial plan and configure the dial plan to use secured mode, both the SIP signaling traffic and the RTP media channels are encrypted. A secure signaling media channel that uses Secure Realtime Transport Protocol (SRTP) also uses mutual TLS to encrypt the VoIP data.

You can configure the VoIP security mode either when you're creating a new dial plan or after you create a dial plan using the Exchange Management Console or the Set-UMDialPlan cmdlet. When you configure the UM dial plan to use SIP secured or secured mode, the Unified Messaging servers associated with the UM dial plan will encrypt the SIP signaling traffic or the RTP media channels, or both. However, to be able to send encrypted data to and from a Unified Messaging server, you must correctly configure the UM dial plan, and devices such as IP gateways or IP PBXs must support mutual TLS.

You can use the Get-UMDialPlan cmdlet in the Exchange Management Shell to determine the security setting for a specific UM dial plan. If the VoIP security parameter has been enabled, you can verify that the Microsoft Exchange Unified Messaging service has started in secured mode by checking the application event log to see whether information events numbered 1114 and 1112 have been logged.

Return to top

How Unified Messaging Determines Security Mode and Selects Certificates

When the Microsoft Exchange Unified Messaging service starts, it checks the associated UM dial plan and the VoipSecurity parameter setting and identifies whether it should start in a secured or an unsecured mode. If it determines that it must start in a secured mode, it then determines whether it has access to the required certificates. If the Unified Messaging server isn't associated with any UM dial plans, it determines which mode to start in by looking at the StartSecured parameter in the Msexchangeum.config file. This parameter can be set with a value of 0 or 1. A value of 1 starts the Unified Messaging server using encryption to protect the VoIP traffic. A value of 0 starts the server, but encryption won't be used to protect the VoIP traffic. If you want to change the startup behavior of the Unified Messaging server from secured to unsecured or from unsecured to secured, you can associate the server with the appropriate UM dial plans and then restart the Unified Messaging server. You can also change the configuration setting in the Msexchangeum.config configuration file and the restart the Microsoft Exchange Unified Messaging service.

If the Microsoft Exchange Unified Messaging service is started in unsecured mode, it will start correctly. However, make sure that you verify that the IP gateways and IP PBXs are also running in unsecured mode. Also, if you're testing the Unified Messaging server's connectivity in unsecured mode, use the Test-UMConnectivity cmdlet with the -Secured:false parameter.

If the Microsoft Exchange Unified Messaging service is started in secured mode, it queries the local certificate store to find a valid certificate to use for mutual TLS to enable encryption. The service first looks for a valid PKI or commercial certificate and then, if an appropriate certificate isn't found, it looks for a self-signed certificate to use. If no PKI, commercial, or self-signed certificate is found, the Microsoft Exchange Unified Messaging service creates a self-signed certificate to use to start in secured mode. If the Unified Messaging server is starting in unsecured mode, a certificate isn't needed.

All the details of the certificate used to start in secured mode will be logged whenever a certificate is used or if the certificate has changed. Some details logged include the following:

• Issuer Name

• Serial Number

• Thumbprint

The thumbprint is the Secure Hash Algorithm (SHA1) hash and can be used to uniquely identify the certificate used. You can then export the certificate used by the Microsoft Exchange Unified Messaging service to start in secured mode from the local certificate store and then import this certificate on the IP gateways and IP PBXs on your network into the trusted certificate store.

After an appropriate certificate has been found and is used, and no additional changes have occurred, the Microsoft Exchange Unified Messaging service will log an event one month before the certificate being used expires. If you don't make any changes to the certificate during this time, the Microsoft Exchange Unified Messaging service will log an event each day until the certificate expires and each day after the certificate has expired.

When the Unified Messaging server is looking for a certificate to use for mutual TLS to establish an encrypted channel, it will look in the trusted root certificate store. If there are multiple certificates that are valid and are from different issuers, the Unified Messaging server will choose the valid certificate that has the longest time before the certificate will expire. If multiple certificates exist, the Unified Messaging server will choose the certificates based on the issuer and the date that the certificate will expire. The Unified Messaging server will look for a valid certificate in this order:

1. PKI or commercial certificate with the longest expiration period.

2. PKI or commercial certificate with the shortest expiration period.

3. Self-signed certificate with the longest expiration period.

4. Self-signed certificate with the shortest expiration period. A valid commercial, PKI, or self-signed certificate is required. If a valid certificate isn't found, the Unified Messaging server will generate a self-signed certificate. The Unified Messaging server needs a valid certificate to encrypt the VoIP traffic when it's operating in SIP secured or secured mode.

   Important

   When a new certificate is installed on a Client Access server used to encrypt Play on Phone data between the Client Access server and a Unified Messaging server, you must run the IISreset command from a command prompt to load the correct certificate.

Return to top

 © 2010 Microsoft Corporation. All rights reserved.