Error: Data Execution Prevention (DEP)

Expert (104)

06-06-2014, 05:18

I have a problem with OpenMSX since version "openmsx-0.10.2-120-ga7c51bd-windows-vc-x86": When I execute openMSX (catapult.exe or openmsx.exe) the Windows give a message: "Data Execution Prevention (DEP) has shut down openMSX to protect the OS". How I can solve this problem?
My OS is Windows Vista Home Premium - x86 - SP 2.

By PingPong

Enlighted (4140)

06-06-2014, 20:40

turn off DEP, it's switchable

By Grauw

Ascended (10772)

06-06-2014, 20:46

Is that a solution? DEP exists for a reason, it’s a security feature.

By Manuel

Ascended (19469)

06-06-2014, 20:53

Thanks for reporting! We'll investigate....

By Manuel

Ascended (19469)

09-06-2014, 11:22

Please try the latest build.

By Paulo Volkova

Expert (104)

11-06-2014, 00:37

Manuel, I tested "openmsx-0.10.2-142-g5d51ce3-windows-vc-x86" version and now works great! No DEP error!
Thank you very much.

By Vampier

Prophet (2413)

11-06-2014, 05:31

must have been a false positive

By Manuel

Ascended (19469)

11-06-2014, 10:48

No, it was actually crashing (on Windows XP, which does not have DEP), but Wouter fixed a crash situation which was apparently what Paulo ran into.

(Data Execution means that the code path is getting into a data section, so it's definitely a bug in the code, causing crashes normally. It's nice that the OS prevents this, because executing data is potentially easily exploitable.)

By TheSpecialist

Expert (113)

11-06-2014, 11:50

It's actually the CPU that prevents this. Without an NX bit featured CPU, the OS is powerless against such things.

By Manuel

Ascended (19469)

11-06-2014, 15:11

Yes, you're right, but I guess the OS has to use it (and it seems XP doesn't).

By mars2000you

Enlighted (6499)

11-06-2014, 15:21

As explained on Wiki, DEP has been introduced on Windows in 2004 with Windows XP Service Pack 2. By default, the protection is not full and can't work with some particular x86 processors :

Quote:

If an x86 processor supports DEP, and the system's BIOS supports it and it has been enabled, either by the manufacturer or the user,[5] then the NX features are turned on in Windows, on a limited "OptIn" basis.[6] (PAE must also be enabled in 32-bit Windows.) This setting provides protection only for a limited set of Windows system and binary files. To achieve full protection, the user must choose either "OptOut" ("turn on DEP for all program services except those I select"), covering all programs and processes not specifically exempted, or "AlwaysOn", covering all. These are configurable through the System Properties interface.[7] If DEP is not supported by the particular x86 processor used, no protection is given. Intel's IA-64 architecture also has a form of NX, supported by Windows.

http://en.wikipedia.org/wiki/Data_Execution_Prevention