Performance Tuning Guidelines for Microsoft Services for Network File System

  • Article

Abstract

This document describes important tuning parameters and settings that can result in improved performance for Microsoft Network File Systems running on your Microsoft® Windows Server™ 2003 system. Each setting and its potential effect are described to help you make an informed judgment about its relevance to the system, workload, and performance goals.

*

On This Page

Introduction Introduction
Performance Tuning for Microsoft Services for Network File System Performance Tuning for Microsoft Services for Network File System
Performance Tuning for Networking Performance Tuning for Networking
Performance Tuning for Storage Performance Tuning for Storage
Performance Tuning for File Servers Performance Tuning for File Servers
Benchmarking File Server Workload (Spec SFS) Benchmarking File Server Workload (Spec SFS)
Related Links Related Links

Introduction

Microsoft® Services for Network File System (NFS) running on Microsoft® Windows Server™ 2003 should perform very well out of the box for most customer workloads. However, it is possible to tune the server settings and see incremental performance gains, especially when the nature of the workload does not vary much over time.

The most effective tunings take into account the hardware, the workload, and the performance goals. This document describes important tuning parameters and settings that can result in improved performance. Each setting and its potential effect are described to help you make an informed judgment about its relevance to the system, workload, and performance goals.

This document assumes the Microsoft Services for Network File System that is included in Services for UNIX 3.5. While most of this guide is also useful for earlier versions of Microsoft Services for Network File System, some tuning parameters may not be applicable.

Note: Registry settings and tuning parameters may have changed significantly from Microsoft® Windows® 2000 Server to Windows Server 2003. Please keep this in mind as you tune your server—using earlier or out-of-date tuning guidelines may produce unexpected results.
As always, care must be taken when manipulating the registry directly. If you must edit the registry, back it up first.

Performance Tuning for Microsoft Services for Network File System

This section gives an overview of the Microsoft Services for Network File System architecture and the various settings that may affect its performance. For specific guidelines relating to the Spec SFS benchmark, see the section entitled “Benchmarking File Server Workload (Spec SFS).”

Microsoft Services for Network File System Server Model

The Microsoft Services for Network File System model consists of two entities, the client and the server. The client receives requests for files located on the server through the redirector (rdbss.sys and NFS mini-redirector nfsrdr.sys). It uses the NFS protocol to send its request through TCP/IP. The server receives multiple requests from the clients through TCP/IP and routes the requests to the local file system (ntfs.sys), which accesses the storage stack.

Figure 1   The Microsoft Services for Network File System Model for Client-Server Communication

Figure 1   The Microsoft Services for Network File System Model for Client-Server Communication

Tuning Parameters for Network File System (NFS)

The following registry tuning parameters could affect performance of file servers running Network File System (NFS).

DefaultNumberOfWorkerThreads

HKLM\System\CurrentControlSet\Services\RpcXdr\Parameters\ (REG_DWORD)

Specifies the number of threads used to handle incoming NFS requests.

OptimalReads

HKLM\System\CurrentControlSet\Services\NfsSvr\Parameters\ (REG_DWORD)

Default is 0. Determines whether or not files are opened for FILE_RANDOM_ACCESS as opposed to FILE_SEQUENTIAL_ONLY. Set this value to 1 to force files to be opened for FILE_RANDOM_ACCESS. This will prevent the file system and cache manager from performing prefetching.

RdWrHandleLifeTime

HKLM\System\CurrentControlSet\Services\NfsSvr\Parameters\ (REG_DWORD)

Controls the lifetime of an NFS cache entry in the file handle cache. Actual lifetime is approximately equal to RdWrHandleLifeTime multiplied by RdWrThreadSleepTime.

RdWrNfsHandleLifeTime

HKLM\System\CurrentControlSet\Services\NfsSvr\Parameters\ (REG_DWORD)

Controls the lifetime of an NFS cache entry in the file handle cache. Actual lifetime is approximately equal to RdWrNfsHandleLifeTime multiplied by RdWrThreadSleepTime.

RdWrNfsReadHandlesLifeTime

HKLM\System\CurrentControlSet\Services\NfsSvr\Parameters\ (REG_DWORD)

Controls the lifetime of an NFS cache entry in the file handle cache. This setting is only applicable when used with NFS_ONLY mode on systems that have more than 1 GB of memory. Actual lifetime is approximately equal to RdWrNfsReadHandlesLifeTime multiplied by RdWrThreadSleepTime.

RdWrThreadSleepTime

HKLM\System\CurrentControlSet\Services\NfsSvr\Parameters\ (REG_DWORD)

Controls the frequency of cleanup thread activity on the file handle cache. Value is in ticks and is non-deterministic. A tick is equivalent to approximately 100 nanoseconds.

FileHandleCacheSizeinMB

HKLM\System\CurrentControlSet\Services\NfsSvr\Parameters\ (REG_DWORD)

Default is 4. Specifies the maximum memory to be consumed by File Handle Cache entries.

LockFileHandleCacheInMemory

HKLM\System\CurrentControlSet\Services\NfsSvr\Parameters\ (REG_DWORD)

Default is 0. Specifies whether the physical pages allocated for the cache size specified by FileHandleCacheSizeInMB should be locked in memory. Setting this value to 1 enables this activity. This setting is only meaningful for systems with more than 1 GB of memory. Pages will be locked in memory (that is, they will not be paged to disk) which will reduce the number of pages available to applications.

MaxIcbNfsReadHandlesCacheSize

HKLM\System\CurrentControlSet\Services\NfsSvr\Parameters\ (REG_DWORD)

Specifies the maximum number of handles for the read data cache per volume. Read cache entries will be created in NFS_ONLY mode only on systems that have more than 1 GB of memory.

SecureHandleLevel

HKLM\System\CurrentControlSet\Services\NfsSvr\Parameters\ (REG_DWORD)

Default is 6. SecureHandleInfo controls the strength of the security on handles given out by NFS Server. You can assign any of the following valid values:

0x0 – Disable all security checks on the NFS handles.

0x1 – Sign the handles given to clients for stronger security.

0x2 – Use IP Address of the client, in addition to other data, to sign the handle.

0x4 – Validate that the parent path of the NTFS field embedded in the handle is actually exported at the time when the handle is exported.

0x6 - Use IP Address of the client in addition to other data, to sign the handle and also validate that the parent path of the NTFS field embedded in the handle is actually exported at the time when the handle is exported.

UseWriteCache

HKLM\System\CurrentControlSet\Services\NfsSvr\Parameters\ (REG_DWORD)

This is a legacy setting. Services for NFS that is included with Network Attached Storage 3.0 no longer relies on this setting to control caching. Rather, caching behavior is controlled by client protocol requests when mounting NFS shares.

If you want to improve write performance of Services for NFS, you should check your OEM manuals for whether the server has a hardware disk controller with configurable write cache. For better write performance, it is recommended that you test with the either one of the following settings:

  • 100 percent write and 0 percent read

  • 50 percent write and 50 percent read

Note that using write caching on the disk controller can result in data loss and/or corruption if the disk subsystem does not have an appropriate battery backup.

NFS-Only Mode

Enhanced NFS performance can be achieved by using the NFSONLY.EXE application. This allows a share to be modified to do more aggressive caching to improve performance. This may be set on a share-by-share basis. NFS-Only mode should not be used on any share that can be accessed by any means other than NFS, because data corruption can occur. However, as much as a 15% improvement has been observed when using an NFS-only share. The syntax of this command is:

NfsOnly <resourcename|sharename> [/enable|/disable]

Resourcename|sharename is the name of the NFS share. The /enable option turns on NFS-Only mode for the specified resource or share, while the /disable option turns off NFS-Only mode for the specified resource or share. The Microsoft Services for Network File System Server service must be restarted for NFS-Only mode to take effect.

Performance Tuning for Networking

The network architecture covers many components, interfaces, and protocols; Figure 2 illustrates some of them. The following sections present tuning guidelines for some of the components for server workloads.

Figure 2   Network Stack Components

Figure 2   Network Stack Components

The network architecture is layered. The layers, as you read from the bottom to the top of Figure 2, can be broadly divided into the following categories:

  • The network driver and Network Driver Interface Specification (NDIS). These are the lowest layers. NDIS exposes interfaces for the driver below it and for the layers above it; for example, TCP/IP.

  • The protocol stack. This implements protocols such as TCP/IP and UDP/IP. These layers expose the transport driver interface TDI  for layers above them.

  • System Drivers. These are typically TDI clients and expose interfaces to user-mode applications.

Tuning for network-intensive workloads can involve tuning for each of the layers. Some of the tunings are described in the remaining sections of this white paper.

Choosing a Network Adapter

Network-intensive applications need high-performance network adapters. This section covers some considerations for choosing network adapters.

WHQL Certification

Choose a network adapter with Microsoft Windows Hardware Quality Labs (WHQL) certification.

Offload Capabilities

Offloading tasks can help lower CPU usage on the server, improving overall system performance. The Microsoft TCP/IP transport can offload one or more of the following tasks to a network adapter that has the appropriate task-offload capabilities:

  • Checksum tasks. *** *** The TCP/IP transport can offload the calculation and validation of IP and TCP checksums for sends and receives.

  • IP security tasks. The TCP/IP transport can offload the calculation and validation of encrypted checksums for authentication headers (AH) and encapsulating security payloads (ESP). The TCP/IP transport can also offload the encryption and decryption of ESP payloads.

  • Segmentation of large TCP packets. The TCP/IP transport supports large send offload (LSO). With LSO, the TCP/IP transport can offload the segmentation of large TCP packets.

  • Stack offload. The entire network stack can be offloaded to a network adapter that has the appropriate capabilities.

Interrupt Moderation

Some network adapters are capable of moderating how frequently they interrupt the host processors to indicate network activity (or its completion). Some network adapters are also capable of making such decisions in an adaptive manner, taking into account network and host-processor load. Moderating interrupts can often result in reduction in CPU load on the host, but unless interrupt moderation is performed intelligently and adaptively, the CPU savings may come at the cost of increases in latency.

64-bit Capabilities

Network adapters that are 64-bit capable can perform direct memory access (DMA) operations to and from high physical memory locations (above 4 GB).

Copper and Fiber Network Adapters

Copper network adapters offer the same performance as their fiber counterparts, but may be less expensive to purchase. The cost of the transceiver on copper network adapters is lower than it is for fiber..

Dual or Quad Port Network Adapters

These network adapters are good for failover scenarios but typically share a single interrupt among all the ports on the network adapter. Using two single-port network adapters usually yields better performance than using one dual-port network adapter for the same workload.

Tuning the Network Adapter

You can optimize network throughput and resource usage by using network adapter tunings (when available and exposed by the network adapter). Keep in mind that the correct set of tunings depends on the network adapter, workload, host-computer resources, and performance goals.

Enable Offload Features

It is almost always beneficial to turn on network adapter offload features. In some instances, however, the network adapter may not be powerful enough to handle the offload capabilities at high throughput. For example, enabling LSO can lower the maximum sustainable throughput on some network adapters. However, if the reduced throughput is not expected to be a limitation, offload capabilities should be enabled even for such network adapters. Note that some network adapters require offload features for send and receive paths to be independently enabled.

Network Adapter Resources

Several network adapters allow the configuration of resources by the administrator. Receive buffers and send buffers are among the parameters that may be set. Some network adapters actively manage their resources, and there is no need to set such parameters for these network adapters.

Interrupt Moderation

Some network adapters expose buffer coalescing parameters (sometimes separately for send and receive buffers) for control over interrupt moderation. It is important to consider buffer coalescing when the network adapter does not perform adaptive interrupt moderation.

Adjusting TCP Parameters

TCP parameters that can be adjusted for high throughput scenarios are listed in Table 1.

Table 1. TCP Parameters

Parameter

Description

TCPWindowSize

This value determines the maximum amount of data (in bytes) that can be outstanding on the network at any given time. It can be set to any value from 1 to 65,535 bytes by using the following registry entry:

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Tcpip \Parameters\TcpWindowSize (REG_DWORD)

The default for a gigabit interface is set to approximately 65,535 (rounded down to the nearest multiple of full TCP packets), 16,384 for a 100 Mbps link, and 8,192 for all interfaces of lower speeds (for example, modems), again rounded down. Ideally, this value should be set to the product of end-to-end network bandwidth (in bytes/s) and the round-trip delay (in seconds), also referred to as the bandwidth-delay product. This value should be set according to the amount of TCP data expected to be received by the computer.

Window Scaling

On a link with high bandwidth-delay product (for example, satellite links), there may be a need to increase the window size to above 64 K. For that, you need to enable TCP Options as specified in RFC 1,323 by appropriately setting the following registry entry:

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Tcpip\Parameters \Tcp1323Opts (REG_DWORD)

To enable window sizes of greater than 65,535, this registry entry should be set to 1 (one). After this change has been made, the registry entry controlling TCPWindowSize can be set to values larger than 64 K (up to 1 GB).

MaxHashTableSize

This value determines the size of the hash table holding the state of TCP connections. The default value is 128 multiplied by the square of the number of processors in the system. When a large concurrent connection load is expected on the system, set the following registry entry to a high value to improve the performance of the hash table:

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Tcpip \Parameters\MaxHashTableSize (REG_DWORD)

The maximum value is 0x10000 (65,536). It is recommended that you consider using the maximum value for large servers that you expect to carry high connection load. Keep in mind that the table uses nonpaged pool, so do not set too high a value for the parameter if the server does not have much available nonpaged pool or if you do not anticipate a high-connection load.

NumTcbTablePartitions

By default, the table holding TCP connection states has as many partitions as the square of the number of processors. In most cases, the setting is appropriate and results in lowered contention on the tables. However, the default may be too high for 16 or more processors, and may result in too much CPU usage. In that case, set the following registry entry to a value lower than the square of the number of processors:

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Tcpip \Parameters\NumTcbTablePartitions (REG_DWORD)

MaxUserPort

A port is used whenever an active connection is used from a computer. Given the default value of available user mode ports (5,000 for each IP address) and TCP time-wait requirements, it may be necessary to make more ports available on the system. You can set the following registry entry to as high as 0xfffe (65534):

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\Tcpip \Parameters\MaxUserPort

Performance Tuning for Storage

The following sections present tuning guidelines for storage workloads.

Choosing Storage

The most important considerations in choosing storage systems are:

  • Providing necessary storage space, bandwidth, and latency characteristics for current and future needs.

  • Selecting a hardware RAID type and backup procedure that provide the required performance and data recovery capabilities.

Additionally, you should always choose a storage adapter with WHQL certification.

Estimating the Amount of Data to be Stored

When you estimate the amount of data to be stored on a new file server, you need to consider these issues:

  • The amount of data currently stored on any file servers that will be consolidated onto the new file server.

  • If the file server will be a replica member, the amount of replicated data that will be stored on the new file server.

  • The amount of data that you will need to store on the file server in the future.

A general guideline is to plan for faster growth in the future than you experienced in the past. Investigate whether your organization plans to hire a large number of people, whether any groups in your organization are planning large projects that will require extra storage, and so on.

You must also take into account the amount of space used by operating system files, applications, RAID redundancy, log files, and other factors. Table 2 describes some factors that affect file server capacity.

Table 2.   Factors That Affect File Server Capacity

Factor

Storage Capacity Required

Operating system files

At least 1.5 GB. To allow space for optional components, future service packs, and other items, plan to allow an additional 3 GB to 5 GB for the operating system volume.

Paging file

1.5 times the amount of RAM by default (optimal size will depend on working set of all applications being run). If possible, the paging file should be placed on a separate spindle.

Memory dump

Depending on the memory dump file option that you have chosen, the amount of disk space required can be as large as the amount of physical memory plus 1 MB.

Applications

Varies according to the application.. Applications can include antivirus, backup, and disk quota software, database applications, and optional components such as Recovery Console, Services for UNIX, and Services for NetWare.

Log files

Varies according to the application that creates the log file. Some applications allow you to configure a maximum log file size. You must ensure that you have adequate free space to store the log files.

RAID solution

Varies; see Choosing the Raid Level later in this document for more information.

Shadow copies

Ten percent of the volume by default, although increasing this size is recommended.
Storage Array Selection

There are many considerations in choosing a storage array and adapters. The choices include the type of storage arrays being used, including the options shown in Table 3.

Table 3. Options for Storage Array Selection

Option

Description

Fibre Channel or SCSI

  • Fibre Channel allows long glass or copper cables to connect the storage array to the system while providing high bandwidth.

  • SCSI provides very high bandwidth but has cable length restrictions.

HW RAID capabilities

It is important for the storage controllers to offer HW RAID capabilities. RAID levels 0, 1, and 5 are described in Table 4.

Maximum storage capacity

  • Total storage area.

  • Bandwidth at which storage can be accessed is determined by the number of physical disks in the array, speed of controllers, type of disk (for example, SCSI or Fibre Channel), HW RAID, and adapters used to connect the storage array to system.
HW RAID Levels

Most storage arrays provide some HW RAID capabilities, including the RAID options shown in Table 4.

Table 4. Hardware RAID Options

Option

Description

RAID 0

RAID 0 presents a logical disk that stripe disk accesses over a set of physical disks.

  • Overall, this is the fastest HW RAID configuration.

  • This is the least expensive RAID configuration, because data is not duplicated.

  • Unlike RAID 1 and RAID 5, RAID 0 does not provide additional data recovery mechanisms.

RAID 1

RAID 1 presents a logical disk that is mirrored to another disk.

  • RAID 1 is slower than RAID 0 for write operations, because the data needs to be written to two or more physical disks, and the latency is the slowest of the write operations.

  • In some cases, RAID 1 can provide faster read operations than RAID 0 because it can read from the least busy physical disk.

  • RAID 1 is the most expensive in terms of physical disks, because two or more complete copies of the data are stored.

  • RAID 1 is the fastest in terms of recovery time after a physical disk failure, because the second physical disk is available for immediate use. A new mirror physical disk can be installed while full data access is permitted.

RAID 5

RAID 5 presents a logical disk that has parity information written to other disks.

  • RAID 5 uses independent data disks with distributed parity blocks.

  • RAID 5 is slower then RAID 0 for writes, because each logical disk write I/O results in data being written to multiple disks. However, RAID 5 provides additional data recovery capabilities over RAID 0, because data can be reconstructed from the parity.

  • RAID 5 requires additional time (compared to RAID 1) to recovery from a lost physical disk, because the data on the disk needs to be rebuilt from parity information stored on other disks.

  • RAID 5 is less expensive than RAID 1, because a full copy of the data is not stored on disk.

Other

Other combinations of RAID exist including RAID 0+1, Raid 10.
Choosing the RAID Level

Each RAID level is trade-off among the following factors:

  • Cost

  • Performance

  • Availability and reliability

You can determine the best RAID level for your file servers by evaluating the read and write loads of the various data types and then decide how much you are willing to spend to achieve the performance and availability/reliability that your organization requires. Table 5 lists and describes four common RAID levels, their relative costs, performance, availability and reliability, and their recommended uses.

Table 5. RAID Trade-Offs

 

RAID-0

Striped

RAID-1

Mirrored

RAID-5

Striped with Parity

RAID-0+1

Striped Mirrors

Minimum number of disks

2

2

3

4

Usable storage capacity

100%

50%

N-1/N

where N is the number of disks

50%

Fault tolerance

None. Losing a single disk causes all data on the volume to be lost.

Can lose multiple disks as long as a mirrored pair isn’t lost.

Can tolerate the loss of one disk.

Can lose multiple disks as long as a mirrored pair is not lost. Varies according to the number of mirrored pairs in the array. 

Read performance

Generally improved by increasing concurrency.  

Good read performance

Generally improved by increasing concurrency.

Improvement from increasing concurrency and dual sources for each request.

Write performance

Generally improved by increasing concurrency.

Worse than JBOD (between 20% and 40% for most workloads)

Poor unless full-stripe writes (large requests) Can be as low as ~25% of JBOD (4:1 requests).

Can be better or worse depending on request size, hot spots (static or dynamic), and so on.

Best uses

Temporary data only

Operating system

log files

  • Operating system

  • User and shared data

  • Application files

  • Operating system

  • User and shared data

  • Application files

  • Log files

If you use more than two disks, RAID 0+1 is almost always a better solution than RAID 1.

When determining the number of disks that should be included in RAID 0, RAID 5, and RAID 0+1 virtual disks, consider the following information:

  • Performance increases as you add disks.

  • Reliability, in terms of mean time to failure (MTTF) of two disks, decreases as you add disks for RAID 5 or a single disk for RAID 0.

  • Usable storage capacity increases as you add disks, but cost also increases.

  • Stripe unit size is important. Software solution is fixed at 64 KB. Hardware solutions may range from 4 KB to 1 MB. Ideal stripe unit size maximizes the disk activity without unnecessarily breaking up requests (so that multiple disks are required to service a single request). For example:

    • One stream of sequential requests (large) on JBOD would keep only one disk busy at a time. To keep all disks busy, the stripe unit needs to be equal to 1/N where N is the request size.  

    • For S streams of small random requests, if S is greater than the number of disks, and if there are no hotspots, striping will not increase performance. However, if there are hotspots, the stripe unit size needs to maximize the chance that a request will not be split, while minimizing the chance of a hotspot falling entirely within one or two stripe units. You might pick a low multiple of the request size, like 5X or 10X, especially if the requests are on some boundary (for example, 4 KB or 8 KB).

    • For fewer streams than disks, you need to split the streams so that all disks are kept busy. Interpolate from the previous two examples. For example, if you have 10 disks and 5 streams, split each request in half (use a stripe unit size equal to half the request size).

Determining the Volume Layout

Whenever possible use separate volumes for each data type. For example, use one volume for the operating system and paging space, and one or more volumes for shared user data, applications, and log files.

Place different data types in separate volumes on different virtual disks. Using separate virtual disks is especially important for any data types that create heavy write loads, such as log files, so that a single set of disks (that compose the virtual disk) can be dedicated to handling the disk I/O created by the updates to the log files.

To gain some performance benefits while minimizing cost, it is often useful to combine different data types in one or more volumes on the same virtual disks. A common method is to store the operating system and paging space on one virtual disk and the user data, applications, and log files in one or more volumes on the remaining virtual disk.

Interrupt Moderation

Some storage adapters are capable of moderating how frequently they interrupt the host processors to indicate disk activity (or its completion). Moderating interrupts can often result in reduction in CPU load on the host, but unless interrupt moderation is performed intelligently; the CPU savings may come at the cost of increases in latency.

Table 6. Options for Interrupt Moderation

Device

Description

64-bit capabilities

Adapters that are 64-bit-capable can perform DMA operations to and from high physical memory locations (above 4 GB).

Copper and fiber (glass) adapters

Copper adapters generally perform the same as their fiber counterparts, and both copper and fiber are available on some fibre channel adapters. Some environments are better suited for either copper or glass adapters.

Dual or quad port SCSI adapters

Some SCSI adapters provide 2 or 4 SCSI buses on a single adapter card. This is often necessary due to SCSI limitations on the number of disks that can be connected to a SCSI bus. Fibre channel disks generally do not have limits on the number of disks connected to an adapter.  

You can adjust the following registry parameters for high throughput scenarios.

CountOperations

This parameter allows you to turn off system and process level I/O counters. This counter affects system and disk counting of disk and network I/O requests. Physical and logical disk counters—in addition to network interface, IP and TCP counters—are not affected by this parameter. It is useful to turn off the process and system counters by using this registry parameter on systems where there is a measurable cost associated with counting I/O at the process and system level but where I/O rates can still be analyzed at the physical, logical, network interface, IP and TCP levels. To turn off the process and system I/O counters, you need to create a registry value (and I/O System key if one doesn’t already exist) and set the value to 0 (REG_DWORD) in the following registry entry:

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Session Manager\I/O System\CountOperations.

A reboot is required for this setting to take effect. Process and system counters can be turned on again either by setting CountOperations to 1 or by removing the CountOperations registry entry.

NumberOfRequests

This parameter allows you to specify the number of SRBs created for a given adapter. This improves performance and allows Windows to give more disk requests to a logical disk, which is most useful for HW RAID adapters that have concurrency capabilities, because each logical disk consists of multiple physical disks. However, the default setting is often less than optimal for many high-speed HW RAID disk arrays. Overall disk array performance can be improved by setting NumberOfRequests to a value in the range of 32 to 96 (decimal). Set the following registry entry:

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\MINIPORT_ADAPTER\Parameters \DeviceN\NumberOfRequests (REG_DWORD)

Replace miniport_adapter with the specific adapter name. Make an entry for each device, and in each entry replace DeviceN with Device1, Device2, and so forth, depending on the number of devices you are adding. A reboot is required for this setting to take effect. For example, for two Emulex LP9000 miniport adapters whose miniport driver name is lp6nds35, you would create the following registry entries set to 96:

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\lp6nds35\Parameters\Device0\NumberOfRequests
HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\lp6nds35\Parameters\Device1\NumberOfRequests

DontVerifyRandomDrivers

This parameter prevents the driver verifier from randomly verifying drivers for debugging. To disable the driver verifier, set a value of 1 (REG_DWORD) for the following registry entry:

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Session Manager\Memory Management\DontVerifyRandomDrivers.

Performance Tuning for File Servers

General Considerations

It is important to select the right hardware to satisfy the expected file server load, keeping in mind average load, peak load, capacity, growth plans, and response times. Hardware bottlenecks will limit the effectiveness of software tuning.

Consider the issues listed in Table 7 when selecting hardware and setting up the operating system.

Table 7. Selecting the Right Hardware for Performance

Issue

Recommendation

Number, type, and speed of processors

  • Larger L2 processor caches will provide better performance.

  • Two CPUs will not be as fast as one CPU that is twice as fast.

Amount of physical memory (RAM)

When your computer is running low on memory and more is needed immediately, Windows Server 2003 uses hard drive space to simulate system RAM. This is known as virtual memory, and is often called the paging file.

  • Try to avoid having a pagefile on the same drive as the operating system files.

  • Avoid putting a pagefile on a fault-tolerant drive, such as a mirrored volume or a RAID-5 volume. Pagefiles don't need fault-tolerance, and some fault-tolerant systems suffer from slow data writes because they write data to multiple locations.

  • Do not place multiple pagefiles on different partitions on the same physical disk drive.

Number, type, and speed of network adapters

  • The network adapter should not be a bottleneck. Newer network adapters can offload some server functions and help performance.

  • Make sure all network adapter settings are optimal.

  • Bind each network adapter to a CPU (the method depends on the number of network adapters, the number of CPUs and the number of ports per network adapter).

  • For details, see Performance Tuning for Microsoft Services for Network File System earlier in this document.

Type of disk controller, number of physical disks and their overall capacity

  • File Servers cache frequently accessed files in memory. However, files that are not accessed frequently must come from disk. Handling large amounts of data with a high number of requests to a high number of files require good disk performance (RAID controller connected to a large number of disks).

  • Keep the system page file, the operating system and the data on separate physical disks if possible.

  • Make sure the allocation unit size is appropriate for the size of the volume.

General Tuning Parameters for File Servers

The following registry tuning parameters could affect performance of file servers.

SystemPages

HKLM\System\CurrentControlSet\Control\SessionManager\MemoryManagement\ (REG_DWORD)

Default is 0. Defines the number of system page table entries that are reserved for mapping I/O buffers and other information into the system address space. Each system page table entry maps one page. Setting this value to 0 allows the system to calculate an optimal number of page table entries based on the platform type and the amount of memory available to the system. Setting this value to 0xFFFFFFFF maximizes the number of system pages and the system creates the largest number of page table entries possible within physical memory. Any other value specifies the number of reserved page table entries and prevents the system from adjusting this value.

LargeSystemCache

HKLM\System\CurrentControlSet\Control\SessionManager\MemoryManagement\ (REG_DWORD)

Default is 1. Determines whether the system maintains a standard size or a large size file system cache, and influences how often the system writes changed pages to disk. Increasing the size of the file system cache generally improves server performance, but it reduces the physical memory space available to applications and services. Similarly, writing system data less frequently minimizes use of the disk subsystem, but the changed pages occupy memory that might otherwise be used by applications.

PagedPoolSize

HKLM\System\CurrentControlSet\Control\SessionManager\MemoryManagement\ (REG_DWORD)

File cache space and paged pool space share a common area in system virtual address. Limiting the paged pool allows for a larger system cache, which causes more content to be cached and allows faster serving of files.

AdditionalDelayedWorkerThreads

HKLM\System\CurrentControlSet\Control\SessionManager\Executive\ (REG_DWORD)

Increases the number of threads in the Delayed Work Queue. Threads in this queue have a lower priority and therefore a higher latency because they must compete with other processing for CPU time. Increasing this value may give better performance in some situations.

NtfsDisable8dot3NameCreation

HKLM\System\CurrentControlSet\Control\FileSystem\ (REG_DWORD)

Default is 0. This parameter determines whether NTFS generates a short name in the 8.3 (DOS) naming convention for long file names and for file names that contain characters from the extended character set. If the value of this entry is 0, files can potentially have two names: the name that the user specifies and the short name that NTFS generates. If the name the user specifies conforms to the 8.3 naming convention, NTFS does not generate a short name.

Changing this value does not change the contents of a file, but it avoids the short-name attribute creation for the file. It also changes the way NTFS displays and manages the file.

NtfsDisableLastAccessUpdate

HKLM\System\CurrentControlSet\Control\FileSystem\. (REG_DWORD)

By default, this registry key is not created.

If you have an NTFS volume with a high number of folders or files, and a program is running that briefly accesses each of these in turn, the I/O bandwidth used to generate the Last Access Time updates can be a significant percentage of the overall I/O bandwidth. To increase the speed of access to a folder or file, you can set NtfsDisableLastAccessUpdate to 1 to disable updating of the Last Access Time. After you use this command and restart the computer, the Last Access Time is no longer updated. If you create a new file, the Last Access Time remains the same as the File Creation Time.

NumTcbTablePartitions

HKLM\system\CurrentControlSet\Services\Tcpip\Parameters\. (REG_DWORD)

By default this key is not created.  

This parameter controls the number of TCP control block (TCB ) table partitions. The TCB table can be partitioned to improve scalability on multiprocessor systems by reducing contention on the TCB table. The value should be a power of two, that is, 2, 4, 8, 16, 32, and so on. On multiprocessor systems, change the number of partitions to four times the number of processors in your system.

TcpAckFrequency

For Gigabit cards:

HKLM\system\CurrentControlSet\Services\Tcpip\Parameters\Interfaces

For each Gigabit adapter, add:

TcpAckFrequency (REG_DWORD) = 13 (decimal)

By default this entry is not in the registry. If only acking data and not any control packets, ack once every 13 packets, instead of the default of two. This helps reduce packet processing costs for the Network Stack in the case of large writes (uploads) from the client into the server.

For FastEthernet cards:

HKLM\system\CurrentControlSet\Services\Tcpip\Parameters\Interfaces

For each FastEthernet adapter, add:

TcpAckFrequency (REG_DWORD) = 5 (decimal)

By default, this entry is not in the registry. If only acking data and not any control packets, ack once every five packets, instead of the default of two. This helps reduce packet processing costs for the Network Stack in the case of large writes (uploads) from the client into the server.

Interrupt Affinity

Interrupt affinity means binding of interrupts from a specific device to specific processor(s) in a multiprocessor server. This enforces running the ISR and DPC routines on the said processor(s). Because network connections and file server sessions all stay on the same network adapter, binding interrupts from the network adapter to a processor allows for processing of incoming packets (server message block (SMB) requests, data) on a specific set of processors, improving locality and scalability. You cannot configure affinity on single-processor computers.

The Interrupt-Affinity Filter (IntFiltr) tool allows you to change the CPU-affinity of the interrupts in a system.

Using this utility, you can direct the interrupts of any device to a specific processor or set of processors (as opposed to always sending interrupts to any CPU in the system). Note that different devices can have different interrupt-affinity settings. This utility will work on any server running Windows Server 2003, regardless of the processor or interrupt controller that is used.

Benchmarking File Server Workload (Spec SFS)

Spec SFS is a benchmark program that lets you measure the performance of Network File System file servers as they handle network file requests from clients. Spec SFS provides you with an overall I/O throughput score and average response time for your server and individual scores for the client computers. You can use these scores to measure, analyze, and predict how well your server can handle file requests from clients. See the Standard Performance Evaluation Corporation Web site at https://www.spec.org for additional information.

Registry Tuning Parameters for Spec SFS on Windows Server 2003

The recommendations contained in this section are specifically designed to give optimal performance in the Spec SFS benchmarks. Some of the recommendations may not be appropriate for general NFS file serving. Additionally, some of the recommendations may not be appropriate for all systems.

Common Registry Settings

The registry settings in the following table are applicable to all systems.

Key

Setting

HKLM\System\CurrentControlSet\Control\SessionManager\
MemoryManagement\

PagedPoolSize = 0xFFFFFFFF (hexadecimal) (default=0)

SystemPages = 0

LargeSystemCache = 0

HKLM\System\CurrentControlSet\Control\Session Manager\Executive

AdditionalDelayedWorkerThreads = 16

HKLM\System\CurrentControlSet\Control\FileSystem\

NtfsDisable8dot3NameCreation = 1 (default is 0)  

Add NtfsDisableLastAccessUpdate = 1

By default this registry key is not created.

HKLM\System\CurrentControlSet\Services\RpcXdr\Parameters

DefaultNumberOfWorkerThreads = 16

Note: This value should be configured as 16 per processor up to 64.

HKLM\System\CurrentControlSet\Services\NfsSvr\Parameters

OptimalReads = 1

RdWrHandleLifeTime = 10

RdWrNfsHandleLifeTime = 60

RdWrThreadSleepTime = 60

RdWrNfsReadHandlesLifeTime = 10

SecureHandleLevel = 0 (default is 6)

UseWriteCache = 0 (default is 0)

HKLM\System\CurrentControlSet\Services\Tcpip\Parameters\

Add NumTcbTablePartitions = 8

By default this registry key is not created.

HKLM\System\CurrentControlSet\Services\Tcpip\Parameters\
Interfaces\

Add TcpAckFrequency (REG_DWORD) = 13 (decimal) for each Gigabit network adapter.

By default this registry key is not created. For FastEthernet adapters set this parameter to 5.
Additional System Specific Registry Settings

The registry settings in the following table are applicable only to systems with 2 GB or more of memory.

Key

Setting

HKLM\System\CurrentControlSet\Services\NfsSvr\Parameters

FileHandleCacheSizeInMB = 1038090240

LockFileHandleCacheInMemory = 1

MaxIcbNfsReadHandlesCacheSize = 8000

The registry settings in the following table are applicable only to systems that have 1 GB or more, but less than 2 GB, of memory.

Key

Setting

HKLM\System\CurrentControlSet\Services\NfsSvr\Parameters

FileHandleCacheSizeInMB = 536870912

LockFileHandleCacheInMemory = 1

The registry settings in the following table are applicable only to systems with 512 MB or more, but less than 1 GB, of memory.

Key

Setting

HKLM\System\CurrentControlSet\Services\NfsSvr\Parameters

FileHandleCacheSizeInMB = 0

LockFileHandleCacheInMemory = 0

Best Practices

The following best practices should be used when running the Spec SFS test.

Server Configuration:

  • NFS-Only mode should be used on all NFS shares used during the Spec SFS test.

  • Use the interrupt affinity filter tool to affinitize each network card to a single processor.

  • Do not install Terminal Services, Indexing Services, or other non-default Services on the server. These additional services consume unnecessary system resources.

For the duration of the Spec SFS test run:

  • Make sure the target server does not have any jobs scheduled to run while the test is running.

  • Turn off all non-essential services that consume system resources (for example, w32time).

  • Log off the server machine. Staying logged on consumes unnecessary resources in the system.

  • Do not use screensavers, wallpapers, or other visual effects that consume unnecessary resources in the system.

  • Do not use terminal server to remotely log on to the server machine being tested.

  • Do not use perform, perfmon, sysmon, or any other tools to measure operating system activity or network traffic. Use of any such tools consumes unnecessary resources and strains the network bandwidth.

See the following resources for further information:

For the latest information about Windows Server 2003, see the Windows Server 2003 Web site (https://go.microsoft.com/fwlink/?LinkId=27963).