Assembler Optimizer

Master (165)

10-08-2020, 13:20

@theNestruo
While the peep-hole optimizer in SDCC has awesome syntax and is very powerful, the default rules provided for z80 are ludicrously limited. Probably little more than a proof of concept, and they have not been maintained thoroughly.

I think it is worth to focus the effort on a single application, MDL, to provide this functionality, because the maintenance of two sets of rules may be painful. Also, I think that MDL has the potential to evolve to be more powerful than the peep hole optimizer in SDCC, and maybe do better register allocation (one can dream).

@santiontanon
"So, for any SDCC user reading this, what would be the most useful 'ideal' way in which MDL should work with SDCC output?"
I am a heavy SDCC user, but MDL does not fit strongly my use pattern. I do not care much about used space (that's why I use C). Sometimes, I care about speed (only on HOT functions). When I identify such bottlenecks, I re-implement them in ASM for a 10x speed increase.
To me, I would run MDL if I could add it to my Makefile as an automated step, optimizing the generated asm. I don't care if it outputs syntax for a different assembler. Also, I would like to disable MDL optimizations from within the C file (i.e., by using a #pragma, or a #define) for time sensitive functions, e.g., VDP access.

If MDL gives me a 10% benefit on space or speed, I might take it, but

Van Ped7g

Expert (67)

10-08-2020, 17:04

santiontanon wrote:

So, I think the set of peep rules that come with SDCC is not very comprehensive.

I didn't check SDCC myself too much (I do follow a bit z88dk github, so I have some rough idea, but I didn't try SDCC or z88dk itself, just watching), but I was under impression it is using strictly only identical peep-hole replacements.

While MDL rules contain some rules which slightly affect results and may eventually break the original code in (practically very rare) cases. So I was under impression MDL is since start about being "practical", ie. using also patterns which should work on "common" assembly code, but not strictly identical only. Like giving advice to the programmer which parts of code may be modified, when vetted by the programmer that the modification is correct.

Van santiontanon

Paragon (1832)

10-08-2020, 18:36

Thanks everyone for the discussion!

@ARTRAG: thanks for the link!!! I missed that one

@TheNestruo: haha, ok! then 2 votes in favor of warnings off by default then

@salutte: MDL currently allows you to disable optimizations for certain lines with a special code in a comment: "; mdl:no-opt" by default (but you can change the specific code via a command line flag). However, "block disable optimizations" are not yet supported. I'll make a note of that, since that might allow doing what you suggest. I believe the .asm files generated by SDCC maintain the original code in comments, so, if that's the case, MDL can find the "codes" there and activate/deactivate optimizations in the right places. I'll investigate!

@Ped7g: I don't know 100% for sure, but I think you are right. I think the pattern language of SDCC's optimizer is more restrictive than MDL's one. Hopefully MDL's optimizations do not break the code haha (although some might do it now). You are also right about the optimizations generated by MDL, there is a flag to allow "unsafe optimizations" to be suggested, and if that is activated, some of the optimizations might not be safe, and it's left to the programmer to judge! But if that flag is not on, hopefully all the optimizations are safe (if they are not, then it'll be due to a bug or another haha).

Van santiontanon

Paragon (1832)

17-08-2020, 07:11

I just released a new version, it has all the features I wanted it to have for the pattern-based optimizer, so, I gave it the "1.0" number (although it's still "alpha v1.0", since lots of things are only experimental): https://github.com/santiontanon/mdlz80optimizer/releases/tag...

I focused on two main things this week:

Robustness: I added a bunch of other open source projects to my test suite, and now I have made sure it works for all of these (27!) projects: transball, Tales of Popolon, XSpelunker, XRacing, Phantomas 2.0, threed (glass), VGMPlay (glass), pong (asmsx), mine (asmsx), robots (asmsx), gmonkey (asmsx), World Rally (asmsx), MSX ngine (asmsx), TPMSX-Engine (asmsx), MetalGear (sjasm), Uridium (sjasm), Deep Dungeon Adventure (sjasm), SCCReplayer4c (sjasm), msx pong (sjasm), Trilotracker (sjasm), Trilotracker Re-Player (sjasm), Boogaboo (sjasm), PyramidWarp Enhanced (tniasm), z80rogue (tniasm), zombie near (tniasm), Subcommander (winape), Flappy Bird (pasmo). Just wanted to thank everyone who made their MSX asm projects open source, as that has helped me a lot in testing!
Automatically applying optimizations to source files: with this new feature (activated with "-poapply"), MDL clones all the source files in a project, adding ".mdl.asm" extension, that are identical to the input files, except that the optimizations are applied (even the include statements are modified to include the ".mdl.asm" files). This is different from the "-asm" feature that currently exists, in that "-asm" parses the input completely, resolves all macros, etc. and saves a dialect agnostic single .asm file (that might look very different from the input files). This new "-poapply" feature might be the way to incorporate MDL easily into makefiles like those used for SDCC projects, etc. It's still experimental and I have not tested it much, but I think it expands the way in which MDL can be used. One caveat of this mode is that some optimizations can only be applied once all the macros are expanded, as each different instantiation of a macro might result in different types of optimization depending on the context. Those optimizations cannot be applied in this mode.

So, in summary, MDL can be used in 3 different ways now:

As a linter: usually incorporated directly into an IDE/text editor, just suggesting optimizations that the user does herself.
"-asm": Automatically generating a single .asm file with all macros resolved/optimizations applied.
"-poapply": Generating files identical to the input, but with the optimizations applied.

I think for this coming week I might stop development and dedicate some time to creating maybe some documentation (which is currently non existent), and maybe a youtube video or a blog post explaining how to use MDL, as the number of flags/options/functionalities has grown so fast that it might be hard to understand for others haha. After that I will continue with my long list of pending improvements to the optimizer [:)]

Van thegeps

Paragon (1259)

17-08-2020, 10:01

What incredible work you are doing! Ad soon as Freedom Fighter will be completed (due to received feedbacks) I'll talk about scrolling routine and finally I'll release the code. So I'll try MDL on my project too...
In the meanwhile, congrats Santi!

Van santiontanon

Paragon (1832)

17-08-2020, 19:00

Thanks thegeps!!! Looking forward to reading about your scrolling routine!! It seems to be something quite different from what other people have usually done before!

Van santiontanon

Paragon (1832)

20-08-2020, 07:24

I just uploaded a few videos to Youtube explaining MDL. The first is here: https://www.youtube.com/watch?v=g5aoF4-r4v4&

The other two are in the same channel. If you want to be able to follow the examples, make sure to watch in high resolution, some times Youtube defaults to 360p, and then the text is not readable :)

Van Bengalack

Paladin (804)

22-08-2020, 08:03

Impressive! And by the way, making videos like this, is very smart!

Van santiontanon

Paragon (1832)

22-08-2020, 22:53

Thanks! Not sure if many people will watch them haha, but they were fun to make

Van thegeps

Paragon (1259)

23-08-2020, 00:32

I'll watch them for sure!