Learning About Computers and the Internet
Tips Blog WinXP Internet Computing Downloads Vista/7 Home

Data Execution Prevention (DEP)
Beginning with Windows XP SP2, Microsoft added a security feature called Data Execution Prevention (DEP). It is described here.

What DEP Does

Windows XP service pack 2 was a major upgrade with substantial emphasis on improving security. Among the security measures that were added was a feature called Data Execution Prevention (DEP), which is also in Vista. DEP has both a hardware and a software function. They implement different defenses and are described in more detail below.

The purpose of DEP is to prevent the execution of code from areas of memory that should be reserved for data only. (For the technical-minded these areas are the data segment, the process heap and the program stack). DEP is aimed at the notorious problems of buffer overflows that have plagued Windows for years. (Those who are curious about the rather technical subject of buffer overflows can consult this Wikipedia article.) Exploiting buffer overflows is a favorite method of malware writers and removing, or at least diminishing, this vulnerability is an important security step. Many past major malware problems such as the Blaster and Sasser worms made use of buffer overflow attacks.

Hardware-based DEP

The hardware-assisted part of DEP is implemented through a special feature that must be present in the CPU. Hardware-enforced DEP relies on the processor to mark memory with an attribute that indicates that code should not be executed from that memory. All areas of memory are marked as non-executable unless they are explicitly designated for executable code.  For several years now, both AMD and Intel chips have come with this capability. If you have a computer dating back to about 2005 or earlier, it is possible that DEP support is missing. Also on some computers, you can disable processor support for hardware-enforced DEP in the BIOS but a disabled setting is not generally the default.

Check if hardware DEP is availableCheck if hardware DEP is available

To test if your system has a CPU that can enforce DEP, the following procedure can be used in Windows XP:

  1. Open the Start menu and right-click on My Computer.
  2. Choose "Properties" from the context menu.
  3. In the "System Properties" sheet, click the "Advanced" tab . (You can also use the System dialog in Control Panel to reach System Properties.)
  4. In the "Performance" section, click the "Settings" button
  5. Next, click the tab "Data Execution Prevention".
A dialog similar to the one on the right will open. This example figure shows the result if your system lacks a CPU that supports DEP. The message highlighted in yellow will be present. If that area is blank, then your CPU is presumed to support DEP.

Another way to check makes use of the command line. In either Windows XP or Vista (as administrator) open a command prompt. Then enter the command wmic OS Get DataExecutionPrevention_Available If the output is "TRUE," hardware-enforced DEP is available.

If you are a true geek, you can use the graphical interface provided by the Windows Management Instrumentation Tester (wbemtest). Checking if your CPU can enforce DEP is a multi-step procedure described at this Microsoft link.

Software-enforced DEP

Beginning with Windows XP SP2, the operating system also contains certain software-enforced procedures. This function is in addition to the hardware defenses and does not depend on the CPU. However, It is much more limited than the hardware-enforced function and involves exception-handling mechanisms for certain Windows system files.

Configuring DEP

The default setting for DEP does not include the full security available from hardware-enforced DEP but is limited to Windows system files. A more general coverage is available but was not chosen as the default setting in order to avoid problems with older or poorly written software that uses data to perform executable actions. Back when RAM was scarce, programs sometimes used this recourse to save memory. Current well-written programs avoid using data as executables.

Because it is an important addition to the arsenal of anti-malware defenses, I think that DEP should be enabled for all programs and services. You may still encounter legitimate programs that will be blocked but DEP can be configured to allow specific programs or services to execute.

Turn on DEP for all programs Enabling DEP for all programs

In Windows XP, open the "Performance Options" dialog box by the procedure previously indicated. Then select the radio button, "Turn on DEP for all programs and services except those I select:" as highlighted in yellow in the figure on the right. Click "OK". Note that the computer will have to be restarted before the change takes effect and you will be asked if you want to restart.

In Windows Vista, the dialog box for enabling DEP is the same but the path to it is slightly different from the one given for XP. There are several ways to get there but one procedure is:

  1. Open the Start menu and right-click "Computer"
  2. Choose "Properties" from the context menu.
  3. Choose "Advanced system settings" from under "Tasks" in the left pane.
  4. Approve the User Account Control query (You will have to be an administrator).
  5. Click the button "Settings" in the Performance section.
  6. Click the tab "Data Execution Prevention"
Note that 64-bit versions of Windows automatically enforce DEP for all programs.

Exempting programs and services from DEP

It is possible to configure DEP settings for individual programs. If you encounter a legitimate program that conflicts with DEP, it can be put on a list of those where DEP will not be applied. The only problem may be finding the executable file that causes the conflict. Click the "Add" button in the "Performance options" dialog box and a browse window will open that allows you to choose the executable that you wish to exclude from DEP. Once the file is listed, click "OK", The figure above shows an example file highlighted in red. (This particular file was chosen for illustration and is not an actual conflicting file.)

DEP switches in the Windows XP boot file Boot.ini

Global DEP settings for Windows XP can be made with a switch called "no execute" in the Boot.ini file. (Details about Boot.ini are on another page on this site.) Configuring via Boot.ini is convenient for systems administrators since it can be scripted. Two additional configuration settings not present in the graphical interface already discussed are available. Thus, there are four DEP configuration settings possible in Boot.ini, corresponding to.four possible values for the switch "/no execute". Unfortunately, the notation used is confusing; the switch values don't always mean what you might think they mean. Table I describes the four possibilities. The first two are equivalent to the settings already described above. The last two are for systems administrators and override user settings.

Table I. Possible values for switch "no execute"
Configuration Description
OptIn The default value. Limits DEP to Windows system binaries.
OptOut Turns on DEP for all programs and services. (Yes, the name seems contradictory.)
AlwaysOn This setting provides full DEP coverage for the whole system with no exceptions and cannot be changed by the GUI method described previously. For systems administrators.
AlwaysOff This setting turns DEP off for the whole system, regardless of hardware DEP support, and cannot be changed by the GUI method described previously. For systems administrators.

How to Determine What DEP Policies are in Effect

Using the Wmic command-line tool to check if hardware DEP was available was described above. This tool can also be used to determine which of the four configurations or policies described in Table I are in effect. Open a command prompt and enter wmic OS Get DataExecutionPrevention_SupportPolicy The command will return an integer from 0 to 3. The meaning of the output is given in Table II.

Table II. Determining DEP policies
Output Policy in effect
0 AlwaysOff
1 AlwaysOn
2 OptIn (default)
3 OptOut

 << Home page ©2002-2016 Victor Laurie    Home page >>