How to Configure Memory Protection in Windows XP SP2

On This Page

Introduction
Before You Begin
Enabling DEP for all Programs on Your Computer
Enabling the DEP Exception List
Configuring System-wide DEP Options
Related Information

Introduction

Microsoft Windows XP Service Pack 2 (SP2) helps protect your computer against the insertion of malicious code into areas of computer memory reserved for non-executable code by implementing a set of hardware and software-enforced technologies called Data Execution Prevention (DEP). Hardware-enforced DEP is a feature of certain processors that prevents the execution of code in memory regions that are marked as data storage. This feature is also known as No-Execute and Execution Protection. Windows XP SP2 also includes software-enforced DEP that is designed to reduce exploits of exception handling mechanisms in Windows.

Unlike an antivirus program, hardware and software-enforced DEP technologies are not designed to prevent harmful programs from being installed on your computer. Instead, they monitor your installed programs to help determine if they are using system memory safely. To monitor your programs, hardware-enforced DEP tracks memory locations declared as "non-executable". To help prevent malicious code, when memory is declared "non-executable" and a program tries to execute code from the memory, Windows will close that program. This occurs whether the code is malicious or not.

Note: Software-based DEP is part of Windows XP SP2 and is enabled by default, regardless of the hardware-enforced DEP capabilities of the processor. By default software-enforced DEP applies to core operating system components and services.

The default configuration of DEP is designed to protect your computer with minimal impact to application compatibility. However, depending on your DEP configuration, it is possible that some programs might not run correctly. You can use the tasks described in this document to configure DEP on your computer:

• Enable DEP for all programs on your computer

• Add programs to the DEP exception list

• Disable DEP for your entire computer

  IMPORTANT:  The instructions in this document were developed by using the Start menu that appears by default when you install your operating system. If you have modified your Start menu, the steps might differ slightly.

For definitions of security-related terms, see the following:

• "Microsoft Security Glossary" on the Microsoft Web site at https://go.microsoft.com/fwlink/?LinkId=35468

For more information regarding DEP, see the following:

• Microsoft Knowledge Base Article 875352 on the Microsoft Help and Support Web site at https://go.microsoft.com/fwlink/?linkid=35494

Before You Begin

This document provides guidance for configuring DEP on Windows XP SP2.

Note: Hardware-enabled DEP is enabled by default on computers with DEP compatible processors that run Microsoft Windows XP 64-Bit Edition. 64-bit applications will not run from "non-executable" areas of memory. Hardware-enabled DEP cannot be disabled.

Software-enabled DEP on Windows XP SP2 and 32-bit applications running on any processors can be configured to use "executable" or "non-executable" areas of memory.

Enabling DEP for all Programs on Your Computer

The default configuration for hardware and software DEP protects core Windows components and services and has a minimal impact on application compatibility, but you can choose to configure DEP to protect all applications and programs on your computer. If you configure DEP to protect all applications and programs on your computer you will have the benefit of additional protection, but it might lead to additional application compatibility issues. If you configure DEP to protect all applications and programs on your computer, you can exempt individual 32-bit applications from software DEP protection if they have compatibility issues. You cannot disable hardware DEP or exempt 64-bit applications running on 64-bit Windows XP systems with DEP compatible processors.

Requirements to perform this task

• Credentials: You must log on to your computer using an account with local Administrator rights.

Configuring DEP to protect all applications

To enable the DEP for all applications

1. Click Start, and then click Control Panel.

2. Under Pick a category, click Performance and Maintenance.

3. Under or Pick a Control Panel icon, click System.

4. Click the Advanced tab.

   

   Figure 1   System Properties - Advanced tab

5. In the Performance area, click Settings.

   

   Figure 2   Performance Options

6. Click the Data Execution Prevention tab.

   

   Figure 3   Data Execution Prevention tab

7. Select Turn on DEP for all programs and services except for those I select.

8. Click Apply, and then click OK. A dialog box appears and informs you that you must restart your computer for the setting to take effect. Click OK.

Verifying DEP Settings for all Programs Are Applied

To verify DEP settings for all programs are applied

1. Click Start, and then click Control Panel.

2. Under Pick a category, click Performance and Maintenance.

3. Under or Pick a Control Panel icon, click System.

4. Click the Advanced tab.

5. In the Performance area, click Settings and then click Data Execution Prevention.

6. Verify that Turn on DEP for all programs and services except for those I select is selected and then click OK to close Performance Settings.

7. Click OK to close System Properties then close Performance and Maintenance.

Enabling the DEP Exception List

If DEP causes a problem with your applications, a dialog box appears to let you know.

Figure 4   Dialog box that appears if an application has attempted to execute and has encountered a problem with DEP

In cases where DEP causes application failures, Microsoft strongly recommends that you contact the application vendor to determine if a DEP-compatible update is available. Installing such an update is the preferred solution for application compatibility issues with DEP.

If no update is available for your application, follow these steps to access and to configure the Exception List. The Exception List is the list of applications that are excluded from DEP.

Note: The DEP exception list functionality is only available if the DEP configuration is set to protect all programs and services. If you configure your computer to protect only essential Windows components and services, the exception list is unavailable.

Requirements to perform this task

• Credentials: You must log on to your computer using an account with local Administrator rights.

Enabling the DEP Exception List

To enable the DEP exception list

1. Click Start, and then click Control Panel.

2. Under Pick a category, click Performance and Maintenance.

3. Under or Pick a Control Panel icon, click System.

4. Click the Advanced tab.

   

   Figure 5   System Properties - Advanced tab

5. In the Performance area, click Settings.

   

   Figure 6   Performance Options

6. Click the Data Execution Prevention tab.

   

   Figure 7   Data Execution Prevention tab

7. Click Add.

8. Locate and select the executable for the application that is failing, and then click Open.

9. In the warning box, click OK. The selected program now appears in the DEP program area.

10. Click Apply, and then click OK. A dialog box appears and informs you that you must restart your computer for the setting to take effect. Click OK.

Verifying DEP Exception List Settings Are Applied

To verify Memory Protection settings are applied

1. Click Start, and then click Control Panel.

2. Under Pick a category, click Performance and Maintenance.

3. Under or Pick a Control Panel icon, click System.

4. Click the Advanced tab.

5. In the Performance area, click Settings and then click Data Execution Prevention.

6. Verify that the exception list contains the desired programs and then click OK to close Performance Settings.

7. Click OK to close System Properties then close Performance and Maintenance.

Configuring System-wide DEP Options

To make any system-wide changes to DEP for your computer, you must modify a switch to the boot.ini configuration file for the Windows installation that you are currently running. The boot.ini switch is:

• /noexecute = Policy_level

Table 1 lists the options for Policy_level.

Table1 DEP boot.ini policy level options

IMPORTANT: After making any changes in the boot.ini file, you must restart your computer.

WARNING: Microsoft recommends that you do NOT disable software-enforced DEP globally. To do this would make your computer less secure. Hardware-enforced DEP cannot be manually disabled.

Requirements to perform this task

• Credentials: You must log on to your computer as an account with local Administrator rights.

Disabling DEP system-wide using boot.ini  

To disable DEP using boot.ini

1. Click Start, and then click Control Panel.

2. Under Pick a category, click Performance and Maintenance.

3. Under or Pick a Control Panel icon, click System.

4. Click the Advanced tab, and in the Startup and Recovery area, click Settings.  

   

   Figure 8   Startup and Recovery settings

5. In the System Startup area, click Edit.

   

   Figure 9   Boot.ini file in Notepad

6. In Notepad, click Edit and then click Find.

7. In the Find what field, type /noexecute and then click Find Next.

8. In the Find dialog box click Cancel.

9. Replace the policy_level (for example, "OptOut") with "AlwaysOff” (without the quotes).

   WARNING: Be sure to enter the text carefully.

   Note: Your boot.ini file switch should now read:

   /noexecute=AlwaysOff

10. In Notepad, click File and then click Save.

11. Click OK to close Startup and Recovery.

12. Click OK to close System Properties and then restart your computer.

Verifying DEP is Disabled

To verify Memory Protection settings are applied

1. Click Start, and then click Control Panel.

2. Under Pick a category, click Performance and Maintenance.

3. Under or Pick a Control Panel icon, click System.

4. Click the Advanced tab.

5. In the Performance area, click Settings and then click Data Execution Prevention.

6. Verify that the DEP settings are unavailable and then click OK to close Performance Settings.

7. Click OK to close System Properties then close Performance and Maintenance.

Related Information

For more information about Windows XP SP2 memory protection, see the following:

• "Changes to Functionality in Microsoft Windows XP Service Pack 2 - Part 3: Memory Protection Technologies" on the Microsoft TechNet Web site at https://go.microsoft.com/fwlink/?linkid=35495

For more information about Windows XP SP2 security, see the following:

• "Windows XP Security Guide v2 updated for Service Pack 2" on the Microsoft Download Center Web site at https://go.microsoft.com/fwlink/?linkid=35309

• "Windows XP Security Guide Appendix A: Additional Guidance for Windows XP Service Pack 2" on the Microsoft TechNet Web site at https://go.microsoft.com/fwlink/?linkid=35465

For definitions of security-related terms, see the following:

• "Microsoft Security Glossary" on the Microsoft Web site at https://go.microsoft.com/fwlink/?linkid=35468