Tips on running an MSX emulator for deep learning?

Resident (35)

18-09-2016, 17:55

Hi all,
So, I have been thinking about trying some of DeepMind's work (e.g. http://www.theverge.com/2016/6/9/11893002/google-ai-deepmind...) on MSX games! Nothing concrete yet, but was wondering if there was an easy way to run an emulator, get the screen contents on every frame, and send it input programmatically? Do you have any tips for doing that? In the worst case, I could just write a wrapper using Win32Api for example to do the same, but it wouldn't really be the same since you wouldn't be able to control which frame precisely to send input... etc.

Thanks and have a great day.

Aangemeld of registreer om reacties te plaatsen

Van ARTRAG

Enlighted (6845)

18-09-2016, 18:15

Openmsx can be controlled using tcl scripts and tcp-ip

http://openmsx.org/manual/openmsx-control.html

Van msaad

Resident (35)

18-09-2016, 19:02

Fantastic! Thank you, sir!'

EDIT: After looking at the page for a while, it is not very clear to me. Does it actually allow an external app to receive e.g. the pixels on the screen at any given frame? Can I send it commands like "Press space bar"? I may have missed it, but it looked to me like it is for controlling the emulator itself (e.g. change settings... etc)

Van wouter_

Champion (484)

18-09-2016, 19:47

The commands to press/release keys in openMSX are keymatrixdown and keymatrixup.

To receive the screen content there are a few possibilities:
1) You could directly inspect the content of the VRAM and the state of the VDP.
2) You could save each frame as a PNG image and inspect those.
Alternatively (a first version of) your AI could look at the RAM content instead of rendered images. That might be an easier way to extract higher level game properties (like player/enemy x/y positions, etc).

To synchronize your application with openMSX on a per-frame basis you can maybe use breakpoints and the 'after frame' command.

So what you're asking should already be possible with openMSX. Though this is not a use-case we directly considered during openMSX development. So possibly you need something more specific. Just ask and we'll see what we can do.

An alternative to controlling openMSX from an external program is to directly include your code in openMSX itself. OpenMSX is open source and relatively easy to compile/extend yourself. This might make synchronization easier and may have better performance. If you're interested in this option, it might be easiest to discuss details on our IRC channel?

Van santiontanon

Paragon (1636)

19-09-2016, 00:48

Hi @msaad, nice idea!!!

A couple of months ago at a conference during a talk by someone from DeepMind, I was precisely thinking about this! I have played around with ALE, but I'd be much more motivated to work with OpenMSX than with ALE! (even if I also had an Atari as a kid ). If you get anything working, please report back your progress!!

Van Vampier

Prophet (2397)

20-09-2016, 07:07

the self learning (deep thinking) stuff got my interest a few months back when I saw a video of someone who programmed a self learning super Mario bot that actually found a few exploits that TAS runners didn't find yet. I don't have time to dive into this but I'm certainly willing to help with the TCL part if needed. (I wrote quite a few openMSX TCL scripts that are part of openMSX)

Van santiontanon

Paragon (1636)

20-09-2016, 23:06

Hey Vampier!
In their Atari work, Google used a project called ALE (based on an open-source Atari emulator), that did this:
- It supports only a subset of the games (not all of them, since the system relies on games that have a "score" to be maximized)
- For every game, they build-in a small script that brings the emulation right at the point where the game starts (skipping all the intro/main menu/etc.). I think this could be easily done with some save-state in the case of OpenMSX, for example
- ALE then records the pixels in the screen and the current score of the game and makes them available frame by frame to the AI, and then AI sends back "actions" (i.e., joystick movement and button pressing), which is the injected into the emulator. (for every game, someone had to figure out where in memory is the game score stored)
- I think (but this I'd have to check), that ALE allows the AI to control the speed of the game (i.e., the AI asks for the pixels of the screen, and then the game is paused until the AI says "now move on to the next game frame"). This is useful if you are testing an AI that needs to do a lot of computation in between frames.

ALE is built by directly extending the code base of the ATARI emulator, but perhaps an easier solution just using TCL scripts would be viable for OpenMSX (I am completely ignorant of what can be done with the TCL scripts in OpenMSX, so no idea! hahaha )

Van Vampier

Prophet (2397)

21-09-2016, 01:04

Santi: TCL can do a lot if you can do it on a real MSX you can do it in TCL... I even had a HTTP server in TCL that provided a web front end from openMSX.

this is the video i talked about earlier: https://www.youtube.com/watch?v=qv6UVOQ0F44

Van ppeccin

Champion (376)

21-09-2016, 01:28

Take a look at http://webmsx.org, very easy to interact to directly in Javascript

Van Vampier

Prophet (2397)

21-09-2016, 16:41

ppecchin: I really love webmsx but openMSX has a bit more functionality that webmsx has (for now )

I would really like do start on the project but I think I will run out of time before the first line of code.

Van santiontanon

Paragon (1636)

22-09-2016, 09:38

The first question I'd have is this: imagine I'm building an AI using Java, C++ or Python. How would I communicate with the openMSX TCL script to get a game screenshot or to send actions? a socket?