Understanding Storage Configuration

  • Artikel

 

Applies to: Exchange Server 2010 SP3, Exchange Server 2010 SP2

Understanding storage options and requirements for the Mailbox server role in Microsoft Exchange Server 2010 is an important part of your Mailbox server storage design solution. For additional information about other key aspects of the design process, see Mailbox Server Storage Design.

Contents

Storage Architectures

Physical Disk Types

Best Practices for Supported Storage Configurations

Storage Architectures

The following table describes supported storage architectures, and provides best practice guidance for each type of storage architecture, where appropriate.

Supported storage architectures

Storage architecture Description Best practice

Direct-attached storage (DAS)

DAS is a digital storage system directly attached to a server or workstation, without a storage network in between. For example, DAS transports include Serial Attached Small Computer System Interface (SCSI) and Serial Attached Advanced Technology Attachment (ATA).

Not available.

Storage area network (SAN): Internet Small Computer System Interface (iSCSI)

SAN is an architecture to attach remote computer storage devices (such as disk arrays and tape libraries) to servers in such a way that the devices appear as locally attached to the operating system (for example, block storage). iSCSI SANs encapsulate SCSI commands within IP packets and use standard networking infrastructure as the storage transport (for example, Ethernet).

Don't share physical disks backing up Exchange data with other applications.

Use dedicated storage networks.

Use multiple network paths for stand-alone configurations.

SAN: Fibre Channel

Fibre Channel SANs encapsulate SCSI commands within Fibre Channel packets and generally utilize specialized Fibre Channel networks as the storage transport.

Don't share physical disks backing up Exchange data with other applications.

Use multiple Fibre Channel network paths for stand-alone configurations.

Follow storage vendor's best practices for tuning Fibre Channel host bus adapters (HBAs), for example, Queue Depth and Queue Target.

Return to top

Physical Disk Types

The following table provides a list of supported physical disk types and provides best practice guidance for each physical disk type where appropriate.

Supported physical disk types

Physical disk type Description Supported or best practice

Serial ATA (SATA)

SATA is a serial interface for ATA and integrated device electronics (IDE) disks. SATA disks are available in a variety of form factors, speeds, and capacities.

In general, choose SATA disks for Exchange 2010 mailbox storage when you have the following design requirements:

  • High capacity

  • Moderate performance

  • Moderate power utilization

Supported:  512-byte sector disks for Windows Server 2008 and Windows Server 2008 R2. In addition, 512e disks are supported for Windows Server 2008 R2 with the following:

  • The hotfix described in Microsoft Knowledge Base article 982018, An update that improves the compatibility of Windows 7 and Windows Server 2008 R2 with Advanced Format Disks is available.

  • Windows Server 2008 R2 with Service Pack 1 (SP1) and Exchange Server 2010 SP1.

Support requires that all copies of a database reside on the same physical disk type. For example, it is not a supported configuration to host one copy of a given database on a 512-byte sector disk and another copy of that same database on a 512e disk. Also be aware that 4-kilobyte (KB) sector disks are not supported for any version of Microsoft Exchange and 512e disks are not supported for any version of Exchange prior to Exchange Server 2010 SP1.

Best practice: Consider enterprise class SATA disks, which generally have better heat, vibration, and reliability characteristics.

Serial Attached SCSI

Serial Attached SCSI is a serial interface for SCSI disks. Serial Attached SCSI disks are available in a variety of form factors, speeds, and capacities.

In general, choose Serial Attached SCSI disks for Exchange 2010 mailbox storage when you have the following design requirements:

  • Moderate capacity

  • High performance

  • Moderate power utilization

Supported:  512-byte sector disks for Windows Server 2008 and Windows Server 2008 R2. In addition, 512e disks are supported for Windows Server 2008 R2 with the following:

  • The hotfix described in Microsoft Knowledge Base article 982018, An update that improves the compatibility of Windows 7 and Windows Server 2008 R2 with Advanced Format Disks is available.

  • Windows Server 2008 R2 with Service Pack 1 (SP1) and Exchange Server 2010 SP1.

Support requires that all copies of a database reside on the same physical disk type. For example, it is not a supported configuration to host one copy of a given database on a 512-byte sector disk and another copy of that same database on a 512e disk. Also be aware that 4-kilobyte (KB) sector disks are not supported for any version of Microsoft Exchange and 512e disks are not supported for any version of Exchange prior to Exchange Server 2010 SP1.

Best practice: Physical disk-write caching must be disabled when used without a UPS.

Fibre Channel

Fibre Channel is an electrical interface used to connect disks to Fibre Channel-based SANs. Fibre Channel disks are available in a variety of speeds and capacities.

In general, choose Fibre Channel disks for Exchange 2010 mailbox storage when you have the following design requirements:

  • Moderate capacity

  • High performance

  • SAN connectivity

Supported:  512-byte sector disks for Windows Server 2008 and Windows Server 2008 R2. In addition, 512e disks are supported for Windows Server 2008 R2 with the following:

  • The hotfix described in Microsoft Knowledge Base article 982018, An update that improves the compatibility of Windows 7 and Windows Server 2008 R2 with Advanced Format Disks is available.

  • Windows Server 2008 R2 with Service Pack 1 (SP1) and Exchange Server 2010 SP1.

Support requires that all copies of a database reside on the same physical disk type. For example, it is not a supported configuration to host one copy of a given database on a 512-byte sector disk and another copy of that same database on a 512e disk. Also be aware that 4-kilobyte (KB) sector disks are not supported for any version of Microsoft Exchange and 512e disks are not supported for any version of Exchange prior to Exchange Server 2010 SP1.

Best practice: Physical disk-write caching must be disabled when used without a UPS.

Solid-state drive (SSD) (flash disk)

An SSD is a data storage device that uses solid-state memory to store persistent data. An SSD emulates a hard disk drive interface. SSD disks are available in a variety of speeds (different I/O performance capabilities) and capacities.

In general, choose SSD disks for Exchange 2010 mailbox storage when you have the following design requirements:

  • Low capacity

  • Extremely high performance

Supported:  512-byte sector disks for Windows Server 2008 and Windows Server 2008 R2. In addition, 512e disks are supported for Windows Server 2008 R2 with the following:

  • The hotfix described in Microsoft Knowledge Base article 982018, An update that improves the compatibility of Windows 7 and Windows Server 2008 R2 with Advanced Format Disks is available.

  • Windows Server 2008 R2 with Service Pack 1 (SP1) and Exchange Server 2010 SP1.

Support requires that all copies of a database reside on the same physical disk type. For example, it is not a supported configuration to host one copy of a given database on a 512-byte sector disk and another copy of that same database on a 512e disk. Also be aware that 4-kilobyte (KB) sector disks are not supported for any version of Microsoft Exchange and 512e disks are not supported for any version of Exchange prior to Exchange Server 2010 SP1.

Best practice: Physical disk-write caching must be disabled when used without a UPS.

In general, Exchange 2010 Mailbox servers don't require the performance characteristics of SSD storage.

Factors to Consider When Choosing Disk Types

There are several trade-offs when choosing disk types for Exchange 2010 storage. The correct disk is one that balances performance (both sequential and random) with capacity, reliability, power utilization, and capital cost. The following table of supported physical disk types provides information to help you when considering these factors.

Factors in disk type choice

Disk speed (RPM) Disk form factor Interface or transport Capacity Random I/O performance Sequential I/O performance Power utilization

5,400

2.5-inch

SATA

Average

Poor

Poor

Excellent

5,400

3.5-inch

SATA

Excellent

Poor

Poor

Above average

7,200

2.5-inch

SATA

Average

Average

Average

Excellent

7,200

2.5-inch

Serial Attached SCSI

Average

Average

Above average

Excellent

7,200

3.5-inch

SATA

Excellent

Average

Above average

Above average

7,200

3.5-inch

Serial Attached SCSI

Excellent

Average

Above average

Above average

7,200

3.5-inch

Fibre Channel

Excellent

Average

Above average

Average

10,000

2.5-inch

Serial Attached SCSI

Below average

Excellent

Above average

Above average

10,000

3.5-inch

SATA

Average

Average

Above average

Above average

10,000

3.5-inch

Serial Attached SCSI

Average

Above average

Above average

Below average

10,000

3.5-inch

Fibre Channel

Average

Above average

Above average

Below average

15,000

2.5-inch

Serial Attached SCSI

Poor

Excellent

Excellent

Average

15,000

3.5-inch

Serial Attached SCSI

Average

Excellent

Excellent

Below average

15,000

3.5-inch

Fibre Channel

Average

Excellent

Excellent

Poor

SSD: enterprise class

Not applicable

SATA, Serial Attached SCSI, Fibre Channel

Poor

Excellent

Excellent

Excellent

Return to top

Best Practices for Supported Storage Configurations

This section provides best practice information about supported disk and array controller configurations.

Redundant Array of Independent Disks (RAID) is often used to both improve the performance characteristics of individual disks (by striping data across several disks) as well as to provide protection from individual disk failures. With the advancements in Exchange 2010 high availability, RAID is no longer a required component for Exchange 2010 storage design. However, RAID is still an essential piece to Exchange 2010 storage design for stand-alone servers as well as high availability solutions that require either additional performance or greater storage reliability. The following table provides guidance for the common RAID types that can be used with the Exchange 2010 Mailbox server.

Supported data types for the Exchange 2010 Mailbox server role

Data type Stand-alone: supported or best practice High availability: supported or best practice

OS, system, or pagefile volume

Supported: All RAID types.

Best practice: RAID1/10.

Use a dedicated array group; don't host both system LUN and data LUNs on the same array group.

Supported: All RAID types.

Best practice: RAID1/10.

Use a dedicated array group; don't host both system LUN and data LUNs on the same array group.

Exchange mailbox database (.edb) file volume

Supported: All RAID types.

Best practice: 5,400 or 7,200 disks = RAID1/10 only.

RAID5* = Maximum of 7 disks per array group and array controller high priority scrubbing and surface scanning enabled.

RAID6* = High priority scrubbing and surface scanning enabled.

Supported: All RAID types.

Just a bunch of disks (JBOD) (not RAID) (three or more database copies).

Best practice: 5,400 or 7,200 disks = RAID1/10 only or JBOD.

When lagged, database copies should have two or more lagged copies, or lagged copies should be protected with RAID.

RAID5* = Maximum of 7 disks per array group and array controller high priority scrubbing and surface scanning enabled.

RAID6* = High priority scrubbing and surface scanning enabled.

Exchange mailbox database log volume

Supported: All RAID types.

Best practice: RAID1/10.

Supported: All RAID types.

JBOD (not RAID) (three or more database copies).

Best practice: RAID1/10.

When lagged, database copies should have two or more lagged copies, or lagged copies should be protected with RAID.

*Includes RAID variations such as RAID50 or RAID51 for RAID5

The following table provides guidance about storage array configurations for Exchange 2010.

Supported RAID types for the Exchange 2010 Mailbox server role

RAID type Description Supported or best practice

Disk array RAID stripe size (KB)

The stripe size is the per disk unit of data distribution within a RAID set. Stripe size is also referred to as block size.

Best practice: 256 KB or greater. Follow storage vendor best practices.

Storage array cache settings

The cache settings are provided by a battery-backed caching array controller.

Best practice: 75 percent write cache and 25 percent read cache (battery-backed cache). Follow storage vendor best practices.

Physical disk write caching

The settings for the cache are on each individual disk.

Supported: Physical disk write caching must be disabled when used without a UPS.

The following table provides guidance about database and log file choices.

Database and log file choices for the Exchange 2010 Mailbox server role

Database and log file options Description Stand-alone: supported or best practice High availability: supported or best practice

File placement: database per log isolation

Database per log isolation refers to placing the database file and logs from the same mailbox database onto different volumes backed by different physical disks.

Best practice: For recoverability, move database (.edb) file and logs from the same database to different volumes backed by different physical disks.

Supported: Isolation of logs and databases isn't required.

File placement: database files per volume

Database files per volume refers to how you distribute database files within or across disk volumes.

Best practice: Based on your backup methodology.

Supported: When using JBOD, divide a single disk into two volumes (one for database; one for log stream).

File placement: log streams per volume

Log streams per volume refers to how you distribute database log files within or across disk volumes.

Best practice: Based on your backup methodology.

Supported: When using JBOD, divide a single disk into two volumes (one for database; one for log stream).

Best practice: When using JBOD, single database per log per volume.

Database size

Database size refers to the disk database (.edb) file size.

Supported: Approximately 16 terabytes.

Best practice:

  • 200 gigabytes (GB) or less.

  • Provision for 120 percent of calculated maximum database size.

Supported: Approximately 16 terabytes.

Best practice:

  • 2 terabytes or less.

  • Provision for 120 percent of calculated maximum database size.

Log truncation method

Log truncation method is the process for truncating and deleting old database log files. There are two mechanisms:

  • Circular logging, in which Exchange deletes the logs.

  • Log truncation, which occurs after a successful full or incremental Volume Shadow Copy Service (VSS) backup.

Best practice:

  • Use backups for log truncation (for example, circular logging disabled).

  • Provision for three days of log generation capacity.

Best practice:

  • Enable circular logging for deployments that use Exchange 2010 data protection features.

  • Provision for three days beyond replay lag setting of log generation capacity.

The following table provides guidance about Windows disk types.

Windows disk types for the Exchange 2010 Mailbox server role

Windows disk type Description Stand-alone: supported or best practice High availability: supported or best practice

Basic disk

A disk initialized for basic storage is called a basic disk. A basic disk contains basic volumes, such as primary partitions, extended partitions, and logical drives.

Supported.

Best practice: Use basic disks.

Supported.

Best practice: Use basic disks.

Dynamic disk

A disk initialized for dynamic storage is called a dynamic disk. A dynamic disk contains dynamic volumes, such as simple volumes, spanned volumes, striped volumes, mirrored volumes, and RAID-5 volumes.

Supported.

Supported.

The following table provides guidance on volume configurations.

Volume configurations for the Exchange 2010 Mailbox server role

Volume configuration Description Stand-alone: supported or best practice High availability: supported or best practice

GUID partition table (GPT)

GPT is a disk architecture that expands on the older master boot record (MBR) partitioning scheme. The maximum NTFS formatted partition size is 256 terabytes.

Supported.

Best practice: Use GPT partitions.

Supported.

Best practice: Use GPT partitions.

MBR

An MBR, or partition sector, is the 512-byte boot sector that is the first sector (LBA Sector 0) of a partitioned data storage device such as a hard disk. The maximum NTFS formatted partition size is 2 terabytes.

Supported.

Supported.

Partition alignment

Partition alignment refers to aligning partitions on sector boundaries for optimal performance.

Supported: The Windows Server 2008 default is 1 megabyte (MB).

Supported: The Windows Server 2008 default is 1 MB.

Volume path

Volume path refers to how a volume is accessed.

Supported: Drive letter or mount point.

Best practice: Mount point host volume must be RAID enabled.

Supported: Drive letter or mount point.

Best practice: Mount point host volume must be RAID-enabled.

File system

File system is a method for storing and organizing computer files and the data they contain to make it easy to find and access the files.

Supported: NTFS support only.

Supported: NTFS support only.

NTFS defragmentation

NTFS defragmentation is a process that reduces the amount of fragmentation in Windows file systems. It does this by physically organizing the contents of the disk to store the pieces of each file close together and contiguously.

Supported.

Best practice: Not required and not recommended.

Supported.

Best practice: Not required and not recommended.

NTFS allocation unit size

NTFS allocation unit size represents the smallest amount of disk space that can be allocated to hold a file.

Supported: All allocation unit sizes.

Best practice: 64 KB for both .edb and log file volumes.

Supported: All allocation unit sizes.

Best practice: 64 KB for both .edb and log file volumes.

NTFS compression

NTFS compression is the process of reducing the actual size of a file stored on the hard disk.

Supported: Not supported for Exchange database or log files.

Supported: Not supported for Exchange database or log files.

NTFS Encrypting File System (EFS)

EFS enables users to encrypt individual files, folders, or entire data drives. Because EFS provides strong encryption through industry-standard algorithms and public key cryptography, encrypted files are confidential even if an attacker bypasses system security.

Supported: Not supported for Exchange database or log files.

Not supported for Exchange database or log files.

Windows BitLocker (volume encryption)

Windows BitLocker is a data protection feature in Windows Server 2008. BitLocker protects against data theft or exposure on computers that are lost or stolen, and it offers more secure data deletion when computers are decommissioned.

Supported: All Exchange database and log files.

Supported: All Exchange database and log files. Windows failover clusters require Windows Server 2008 R2 or Windows Server 2008 R2 SP1 and the following hotfix: You cannot enable BitLocker on a disk volume in Windows Server 2008 R2 if the computer is a failover cluster node. Exchange volumes with Bitlocker enabled are not supported on Windows failover clusters running earlier versions of Windows.

For more information about Windows 7 BitLocker encryption, see BitLocker Drive Encryption in Windows 7: Frequently Asked Questions.

Return to top

Windows Disk Timeouts

Starting with Exchange 2010 Service Pack 1 (SP1), intelligence is included to deal with hung I/O. Before Exchange 2010, Exchange reported slow I/O in the event log, but does not take any other action. Exchange 2010 SP1 will actively fail (bugcheck) the server if the hung I/O is affecting active databases on a DAG node.

The new recovery logic in Exchange 2010 SP1 leverages the built-in Windows bugcheck behavior when certain conditions occur. Specifically, when hung IO occurs. The Extensible Storage Engine (ESE) has been updated to detect hung IO and to take corrective action to automatically recover the server.

ESE maintains an IO watchdog thread that detects when an IO has been outstanding for a specific period of time. By default, if an IO for a database is outstanding for more than one minute, ESE logs an event. If a database has an IO that has been outstanding for more than four minutes, ESE logs a specific failure event, if it is possible to do so.

ESE Event 507, 508, 509, or 510 may or may not be logged, depending on the nature of the hung IO. If the nature of the problem is such that the OS volume is affected or the ability to write to the event log is affected, the events are not logged. If the events are logged, the Microsoft Exchange Replication service (MSExchangeRepl.exe) detects that condition, and intentionally cause a bugcheck of Windows by terminating the wininit.exe process. The following table describes the recovery logic behavior in Exchange 2010 SP1 and earlier versions.

Exchange Version I/O Type I/O Time Behavior

Exchange Server 2003

Completed

>60 seconds

Write to Event Log

Exchange Server 2007

Completed

>60 seconds

Write to Event Log

Exchange 2010 RTM

Completed

>60 seconds

Write to Event Log

ESE performs clean-page overwrite on pages affected by slow I/O

Exchange 2010 SP1

In Flight

>60 seconds

Write to Event Log

Exchange 2010 SP1

In Flight

>4 minutes

Terminate wininit.exe process and bugcheck the server

Exchange Server 2010 SP1

Completed

>30 seconds

Write to event log

ESE performs clean-page overwrite on pages affected by slow I/O

Note

In the I/O Type column of the table, In Flight describes a slow I/O operation that has not yet successfully finished. and Completed describes a slow I/O operation that took more than 30 seconds to finish. The concept of detecting slow I/O in-flight operations is new in Exchange 2010. In earlier versions of Microsoft Exchange, the program reported only after the I/O had finished.

We recommend that you do not change the new recovery logic behavior in Exchange 2010 SP1. However, if you must change the new behavior, see New High Availability and Site Resilience Functionality in Exchange 2010 SP1 for more information about how to do this.

The following table outlines the recommended guidance for setting the HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Disk\TimeOutValue registry subkey for servers that are running the Exchange 2010 Mailbox role.

Scenario Recommendation

Direct-Attached Storage

Reduce Windows disk TimeOutValue to 20 seconds

Refer to hardware manufacturer’s guidance

Hardware manufacturer’s guidance takes priority in the event of a clash

SAN-Attached RAID Storage

Reduce Windows disk TimeOutValue to 20 seconds

Refer to hardware manufacturer’s guidance

Hardware manufacturer’s guidance takes priority in the event of a clash

JBOD Storage

Increase Windows disk TimeOutValue to 180 seconds

Refer to hardware manufacturer’s guidance

Hardware manufacturer’s guidance takes priority in the event of a clash

For more information, see the Exchange Server Team Blog article Windows Disk Timeouts and Exchange Server 2010.

 © 2010 Microsoft Corporation. All rights reserved.