Yet another tool to convert MSX Basic written in modern code standards + Syntax Highlight for Sublime Text 3

Page 1/2
| 2

By farique

Rookie (23)

farique's picture

22-05-2019, 22:02

Well, hi.
Here is another way to write MSX Basic the modern way, on a modern editor, and convert it back to the classic format.

MSX Basic Dignified.
https://github.com/farique1/msx-basic-dignified

I know there are already some very good alternatives (Tabmegx, Inliner, NestorPreTer) but I needed the Python exercise (it is horribly done, my very first Python lines of code, be advised) and I wanted something better suited to my normal coding workflow.

Also there is a syntax highlight, a theme and a comment config for Sublime 3 tuned to MBD that works just fine for the stock MSX Basic.
https://github.com/farique1/MSX-Sublime-Syntax

As I said, the thing is pretty much wild and experimental so take it easy.
Cheers.

Login or register to post comments

By Grauw

Ascended (8395)

Grauw's picture

22-05-2019, 23:44

Neat stuff! And I like how you slam a nice tool like that together in under a week’s time.

p.s. Difficulty understanding Git really has nothing to do with age or you as a person Wink.

By farique

Rookie (23)

farique's picture

23-05-2019, 16:28

Thank you, Grauw Smile

By Louthrax

Prophet (2076)

Louthrax's picture

23-05-2019, 21:38

Impressive achievement for a first Python project !

Maybe a next step could be to generate tokenized BASIC ? Just suggesting... there are already existing tools, but IIRC they were not command-line oriented and hard to integrate in a "make-like" build-chain (I'd definitively be interested by that Smile)

By Pencioner

Paladin (933)

Pencioner's picture

23-05-2019, 21:57

@farique great to have such tools developed Smile

@Louthrax for tokenized, is there maybe option for running openmsx (headless? without video output?) and control it via script or so - make it enter a command to load an ascii basic program and save back tokenized? it is a trick but it would definitely do the job and take less effort than making a tokenizer in python (which also could be not completely error prone in the end, because it is quite complicated and there are no guarantee that it will work exactly like a native one). i'm not 100% sure but i have a gut feeling that openmsx has enough of features for this task

By turbor

Champion (425)

turbor's picture

24-05-2019, 10:17

@pencioner Using openmsx headless is possible and would work, the main drawback with this proposed solution is that there would be no feedback in case of errors. For instance lines to long, program to big to fit in memory etc etc
So you would need to check these conditions first and then assume that the emulated MSX will always be possible to parse the input without any error...

By NYYRIKKI

Enlighted (5366)

NYYRIKKI's picture

24-05-2019, 14:58

Few tips, if someone wants to make a new tokenizer:

In openMSX there is also "listing"-script in the scripts folder that pretty well explains how the tokenized program is stored. If the line is too long (>254 visible characters IIRC) the encoded program still works, but you can't edit it anymore in MSX full screen editor or the invisible extra information is lost from end of the line. It should be noted also that the memory pointers in encoded file don't really matter as MSX will calculate them again after loading anyway.

By farique

Rookie (23)

farique's picture

24-05-2019, 17:39

@Louthrax, I have to admit I gave one minute of consideration to this possibility but it is a whole new ball game and, as @Pencioner pointed, it is also a pandora box, there is hope on the bottom but madness to get there Tongue

For the sake of the new guy here (me), apart from being smaller, faster to load (and more compatible?) what are the benefits of a tokenized approach during production?

The way I worked during the tests was to have a two column Sublime (source and converted on each side), openMSX with throttle turned on (to help on the loading speed), dir as disk inserted and F1 set to 'CLS:RUN"convert.bas"+enter'.
I worked on the source, saved, converted, and pressed F1 on openMSX. If I wanted to test or debug something quickly, straight on the emulator, I used the converted window on sublime to find the offending line. The back and forth was very fast.

By NYYRIKKI

Enlighted (5366)

NYYRIKKI's picture

24-05-2019, 18:52

farique wrote:

For the sake of the new guy here (me), apart from being smaller, faster to load (and more compatible?) what are the benefits of a tokenized approach during production?

No, there are no compatibility benefits. The smaller size & faster to load is pretty much all there is... If this would be done correctly then I could imagine that this kind of program could help if someone is planning to put BASIC program inside ROM-file as MSX internal BASIC editor is not really good for this kind of conversion... but I would say this would be very theoretical use case... especially in 2019.

By ricbit

Champion (437)

ricbit's picture

24-05-2019, 20:54

turbor wrote:

@pencioner Using openmsx headless is possible and would work, the main drawback with this proposed solution is that there would be no feedback in case of errors. For instance lines to long, program to big to fit in memory etc etc
So you would need to check these conditions first and then assume that the emulated MSX will always be possible to parse the input without any error...

You can place a breakpoint in basic's error messages and catch the exception using tcl scripts. This way you can get the error reporting you need. I have done that in the code coverage tool for my msx json parser. In my case I placed the breakpoint at the FF07 hook ("Ok" handler), if basic prints anything but Ok then it's considered an error to me.

By ricbit

Champion (437)

ricbit's picture

24-05-2019, 20:54

turbor wrote:

@pencioner Using openmsx headless is possible and would work, the main drawback with this proposed solution is that there would be no feedback in case of errors. For instance lines to long, program to big to fit in memory etc etc
So you would need to check these conditions first and then assume that the emulated MSX will always be possible to parse the input without any error...

You can place a breakpoint in basic's error messages and catch the exception using tcl scripts. This way you can get the error reporting you need. I have done that in the code coverage tool for my msx json parser. In my case I placed the breakpoint at the FF07 hook ("Ok" handler), if basic prints anything but Ok then it's considered an error to me.

Page 1/2
| 2