Frame-by-frame processing (slowdown issue)

By a5kin

Supporter (4)

a5kin's picture

28-02-2016, 12:52

Hi all!

I'm trying to implement AI bot to play MSX games.
In order to train a bot, I need at least following steps to be performed:

  1. Wait until frame is ready
  2. Pause game immediately
  3. Take a screenshot
  4. Process a screenshot
  5. Emulate keys pressed/unpressed
  6. Unpause game
  7. Start all over for the next frame

Of course, I need all the work to be done at maximum speed, or training would take months Smile

Right now, I've managed how to do it, but my solution has a major shortcoming: after ~7000 frames emulation starts to slow down, and after 10000 frames it becomes dramatically slow (5-10 frames/sec).

So, I'm using following command to run emulator:
openmsx -control stdio -machine C-BIOS_MSX1

Then, sending initialization commands to stdin:

cart ./roms/some.rom
set power on
set renderer SDL
set scale_factor 1
set pause on
set throttle off
reverse stop

Then, at each step, sending:

after frame "set pause on; screenshot -raw ./screen.png"
set pause off

Then, waiting until stdout has a reply containing "Screen saved", and processing a frame.

As I said before, this approach results in significant slowdown after time passed.
I also tried to do "after cancel ..." after each screenshot is done, but it doesn't help.

So, I have a couple of questions:
What would cause a slowdown in my approach?
Is there another way to process emulation frame-by-frame?

Thanks in advance!

Login or register to post comments

By ricbit

Champion (438)

ricbit's picture

28-02-2016, 17:17

Do you really need the screenshot? It will be faster to just dump the vram contents, or depending on the game training directly with the ram contents.

By Manuel

Ascended (18233)

Manuel's picture

28-02-2016, 18:55

I don't know the cause of the slowdown (yet), but:
1. indeed, better take analysis of game variables than of a screenshot. But I don't know how your bot is working exactly.
2. No need to enable pause. The commands are done right away, so setting pause on and off has no effect at all.
3. You can of course send these commands via stdin, but you can also just start openMSX with a small Tcl script to set the correct settings (or even save these settings permanently). See the -script cmdline option.
4. How do you check for 'screen saved' and how do you proceed after that exactly?

By a5kin

Supporter (4)

a5kin's picture

28-02-2016, 18:56

ricbit wrote:

Do you really need the screenshot? It will be faster to just dump the vram contents, or depending on the game training directly with the ram contents.

Yes, I thought about dumping VRAM, but not sure how to do it correctly. Can you please suggest a command(s) to do it?

Anyway, the problem with a slowdown stays even if I replace screenshot command with message. Any thoughts why? Would it be because of too many postponed commands? Or too many pauses?

By a5kin

Supporter (4)

a5kin's picture

28-02-2016, 20:32

Manuel wrote:

I don't know the cause of the slowdown (yet), but:
1. indeed, better take analysis of game variables than of a screenshot. But I don't know how your bot is working exactly.
2. No need to enable pause. The commands are done right away, so setting pause on and off has no effect at all.
3. You can of course send these commands via stdout, but you can also just start openMSX with a small Tcl script to set the correct settings (or even save these settings permanently). See the -script cmdline option.
4. How do you check for 'screen saved' and how do you proceed after that exactly?

1. I intent to write a bot that takes a video frames as an input, and nothing more, then decide what keys to press. Something like guys at DeepMind did recently. https://www.cs.toronto.edu/~vmnih/docs/dqn.pdf
2. I need pause to make sure game is not running when NN making its decision. Otherwise, next frame(s) may be ready before previous frame is processed.
3. The reason I'm sending commands via stdin/out is because I need to run them interactively, depending on NN decision.
4. I'm just constantly reading lines from stdout in separate thread, and check them for substring, then activate frame processing.

BTW, here is current version of a Python script I'm using to test frame-by-frame processing, just in case:
http://pastebin.com/q2C6Ukr4

By NYYRIKKI

Enlighted (5889)

NYYRIKKI's picture

28-02-2016, 20:26

a5kin wrote:

Yes, I thought about dumping VRAM, but not sure how to do it correctly. Can you please suggest a command(s) to do it?

I think maybe save_msx_screen-script could be useful for you... You should find it from /share/scripts folder or look here... This makes the data a bit more user friendly to handle as it arranges VRAM content to unified form automatically... at least you can see the correct commands.

I don't know if this is good for this kind of AI you are planning, but at least you have sprite data already separated that in my mind should make things a bit more easy... How ever your goal is way over my head. I would indeed start by monitoring game specific RAM variables instead.

By wouter_

Champion (469)

wouter_'s picture

28-02-2016, 20:31

Hi a5kin,

This was a complexity problem in openMSX. I solved it in this commit. Thanks for reporting this problem!
The fix will be part of the next openMSX release. But if you don't want to wait for that, you can in your openMSX installation replace the file share/scripts/_osd_widgets.tcl with this fixed version.

To dump the content of the VRAM you can use the "dumpvram filename" command. Or if you need more control, you can use the more low-level save_debuggable or debug commands. Though then you have to do sprite rendering yourself. But if you're not careful, your AI can 'see' what's below a sprite, or it can distinguish between two color values that happen to have the same palette settings. So if your goal is have the AI use the same visual input as what a human sees, then dumping the VRAM might be considered cheating.

Wouter

By a5kin

Supporter (4)

a5kin's picture

28-02-2016, 21:38

wouter_ wrote:

This was a complexity problem in openMSX. I solved it in this commit.

Thank you so much, Wouter! No more slowdown with your fix =)

Regarding VRAM dumping, I cannot find "dumpvram" command neither in online doc, nor in my openMSX version (0.12.0-1). Is it a new command and not yet in stable version?

And yes, as an input I need exactly the same picture as human sees. So probably VRAM is cheating.

By wouter_

Champion (469)

wouter_'s picture

28-02-2016, 21:44

Sorry, my mistake. It's called "vramdump" (not "dumpvram").