Assembler Optimizer

Paragon (1628)

23-08-2020, 16:55

Hello Santiontanon,

I still haven't been able to make mdl work with my code.

java -jar mdl.jar main.asm -asm+ annotated.txt
ERROR: parseRestofTheLine: Cannot parse line main.asm#7: VDP=9938  left over tokens: [=, 9938]

The line are:

VDP=9938

if VDP=9938 
	output "..\Binaries\main_v9938.rom"
else
	output "..\Binaries\main_v9990.rom"
endif

Van santiontanon

Paragon (1832)

23-08-2020, 19:44

Hi Metalion, yes, notice that the line you mention is not standard Z80 syntax, and thus, you need to specify which dialect you are using. For example, if you use "sjasm", your call should be:

java -jar mdl.jar main.asm -asm+ annotated.txt -dialect sjasm

Instead of what you used. Currently, the supported dialects are: asmsx, glass, sjasm, tniasm, winape, pasmo, sdcc

Without specifying a dialect, MDL will only parse basic standard Z80 syntax (basically: org, equ, db, dw, assembler instructions, and little else).

Van Metalion

Paragon (1628)

23-08-2020, 20:49

Thanks, but it still does not work (no annotated output from mdl).
It seems to have problems with macros.

WARNING: Redefining symbol macro
WARNING: First defined in macros.asm, 27 as $: macro:    select bank, address
WARNING: Redefined in macros.asm#35: macro      add_hl_a
WARNING: Redefining symbol macro
WARNING: First defined in macros.asm, 27 as $: macro:    select bank, address
WARNING: Redefined in macros.asm#46: macro      a_to 2
ERROR: No op spec matches with operator in line macros.asm#47:  ld      @2,a
; 5
ERROR: Problem including file at main.asm#47:         include "macros.asm"
        ; Librairie de macros

Line 47 in macros.asm are those ones:

macro	a_to 2
	ld	@2,a	; 5
	rla		; 5
	sbc	a	; 5
	ld	@1,a	; 5
endmacro

Van santiontanon

Paragon (1832)

23-08-2020, 22:08

Ok, I see the problem (there's two of them actually)! It all comes down to the fact that sjasm parses a few constructs beyond what is documented. So, it is thanks to trying MDL in different code bases that I discover all these edge cases

The first errors are because in the sjasm manual the "macro" command always appears without a colon afterwards, and thus that's the way I had it implemented. But it seems that it can also have a colon afterwards (which I thought was only used for labels). I'll make a note as it should not be hard to support. I also see another detail in the syntax that is causing the error in line 47. Both these things you point out above should be supported in the next release.

Doing my best here to support all kinds of syntax, but some assemblers (and sjasm in particular) support LOTS of syntax constructs, so, I'm adding them as I encounter them!

[edit: I just committed a fix for these two cases, they were not too hard, it will be included in the next release, maybe later tonight]

Van santiontanon

Paragon (1832)

24-08-2020, 06:23

Just uploaded a new release (alpha 1.1): https://github.com/santiontanon/mdlz80optimizer/releases/tag...

No big new features this time, just bug fixes here and there and a few more optimization patterns. I focused on the videos this week. So, only a small improvement on the actual software :)

Metalion, I hope this one works! Cannot guarantee 100% without having the project to test myself. But at least those issues you posted above should be fixed :)

Van Metalion

Paragon (1628)

24-08-2020, 08:36

A lot of thanks for your work on this very good tool !

But you are right : sjasm does not support semicolons after the macro command. The warning message from mdl makes a semicolon appear after the macro command, but it is not there in my code. I think it comes rather from the structure of the mdl warning message.

WARNING: First defined in macros.asm, 27 as $: macro:    select bank, address

macro	select	bank,address
	ld	a,bank
	ld	(address/4+5000h),a	; mapper ASCII8
endmacro

So the problem must be elsewhere.

Van santiontanon

Paragon (1832)

24-08-2020, 08:47

oh, I see! My bad haha, ok, I'll remove that feature then (in v1.1 linked above, "macro:" would work ). The other problem that I fixed in v1.1 is that I had assumed the keywords would be indented, but this does not need to be the case. So, I relaxed that constraint. That could have been the issue (but if version 1.1 https://github.com/santiontanon/mdlz80optimizer/releases/tag/alphav1.1 that I released today still does not work, then I'll try to look into it a bit more tomorrow! now it's bed time in my time zone! haha)

And no problem at all, thanks a lot for testing and for the patience while I fix all the bugs

Van Bengalack

Paladin (804)

27-09-2020, 08:05

I have tested this on SDCC-dialect. It does not seem to do anything. And when I try to output a file, theres no file created. The output is this:

C:\source\repos\noname\project\Debug\objs>java -version
java version "1.8.0_261"
Java(TM) SE Runtime Environment (build 1.8.0_261-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.261-b12, mixed mode)

C:\source\repos\noname\project\Debug\objs>java -jar mdl.jar ..\..\allsorts.s -po -dialect sdcc
WARNING: Redefining symbol .macro
WARNING: First defined in ..\..\allsorts.s, 79 as $: .macro:    ld____d_iyh
WARNING: Redefined in ..\..\allsorts.s#83: .macro ld____e_iyl
WARNING: Redefining symbol .endm
WARNING: First defined in ..\..\allsorts.s, 81 as $: .endm:
WARNING: Redefined in ..\..\allsorts.s#85: .endm

C:\source\repos\noname\project\Debug\objs>java -jar mdl.jar ..\..\allsorts.s -po -dialect sdcc -asm test.asm
WARNING: Redefining symbol .macro
WARNING: First defined in ..\..\allsorts.s, 79 as $: .macro:    ld____d_iyh
WARNING: Redefined in ..\..\allsorts.s#83: .macro ld____e_iyl
WARNING: Redefining symbol .endm
WARNING: First defined in ..\..\allsorts.s, 81 as $: .endm:
WARNING: Redefined in ..\..\allsorts.s#85: .endm

Also, I see from the docs that hex can have different syntax. SDCC always prepend any number with # (yes decimals too), but making it hex is by prepending "0x", like "#0xFF". Could this be supported?

Van santiontanon

Paragon (1832)

27-09-2020, 18:34

Hi Bengalack! SDCC support is still in development! In the head version in GitHub part of what you are asking is already supported, but it's not yet complete. I took a break over the past few weeks on the project, but will retake it soon. And definitively, full SDCC support is on the way! I am using @salutte's SDCC build script as my test example. More news soon!

Van Grauw

Ascended (10821)

28-09-2020, 02:56

I was looking into compressing “stream of register writes” type music data into a hierarchical structure, and while I am far from finished with the research (currently reading Handbook of Data Compression and LZ78 has peaked my interest so far), I realised that this problem can be generalised to assembly code.

A quick search later and I have found this paper which you may find interesting as well: Analyzing and Compressing Assembly Code.

Basically, recognising repetitive code and optimising it into subroutine calls. Presume for a moment that you can unroll a program fully (disregard conditional calls and loops for a moment, and consider jumps as tail calls), by first transforming code to be fully linear, you can reconstruct it from scratch to optimise the amount of shared code.

This has the potential for much greater space savings than peephole optimisations can achieve.

Quote:

The code compressor has been run on a range of programs, including 157 UNIX system utilities. Different samples might give different results, but, so far, compression ratios have ranged 0-39% with an average of 7%. Compressed code has taken 1-5% more CPU time but as much as 11% less real time, presumably because it loads faster.

Super interesting, and this suffix or Patricia tree is going to be very useful for my purposes. I also see other benefits of this data structure for code analysis.