Gunzip for MSX 1.0 released

Gunzip for MSX 1.0 released

van Grauw op 18-10-2015, 17:56
Onderwerp: Software
Labels: grauw, gunzip, gzip
Talen:

Yesterday Laurens Holst released gunzip, a high-performance gzip (.gz) archive decompressor. Gzip is a relatively common file format to compress single files, with compression ratios comparable to most other archive formats.

You can get it from the downloads page on the project site.

This gunzip implementation is at least 2x as fast as other decompression tools for msx (excluding special-purpose formats like BitBuster). Tests showed that a 720K disk image can be decompressed in 26 seconds on turboR, and 133 seconds on MSX2. Full performance test results are presented in the forum thread, which also contains some technical details about the techniques used to achieve this high performance.

Gunzip is open source and liberally licensed, so you can reuse the code in any way you want. Gzip’s deflate compression algorithm is also used by various other file formats such as zip and png, so hopefully it will come in useful.

The project was started because VGM files for VGMPlay are commonly compressed with gzip. Until now, to play back these .vgz files you had to extract them on PC, but now you can use gunzip to decompress these files on the MSX. Later it will be integrated into VGMPlay.

Relevant link: Gunzip for MSX

Reacties (16)

Van Louthrax

Prophet (2093)

afbeelding van Louthrax

18-10-2015, 18:59

This looks like a major contribution to the Z80 scene, and it started on MSX Cool Thanks Grauw !

Van snout

Ascended (15187)

afbeelding van snout

18-10-2015, 20:58

I cannot put into words how epic this is, particularly when looking at the sheer speed of the decompressor. First ZIP, now gzip. Not a bad year for MSX I'd say.

Van syn

Paragon (1920)

afbeelding van syn

18-10-2015, 21:04

The guy who makes runrar will be a hero for ever! Tongue

Van Grauw

Ascended (8507)

afbeelding van Grauw

18-10-2015, 21:34

The problem with rar and 7zip is that the dictionary sizes are too big (megabytes / gigabytes). Effectively on MSX it means you have to decompress the entire file into memory before writing to disk. Whereas the dictionary size for gzip and zip is just 32K which will fit in main memory just fine without even dealing with memory mapping.

So although they achieve a little higher compression ratios, I don’t think rar and 7z are suitable for MSX due to the memory requirements. There’s no need for them either, as gzip and zip are still extremely popular formats with built-in support in nearly every modern desktop OS.

Van syn

Paragon (1920)

afbeelding van syn

18-10-2015, 22:02

Hehe I was joking, as a play on the names of the msx unpackers released this year Big smile

Van Grauw

Ascended (8507)

afbeelding van Grauw

18-10-2015, 23:59

Hehe yeah, I know, but it was worth mentioning anyway Smile.

Van cbsfox

Champion (327)

afbeelding van cbsfox

19-10-2015, 06:28

Congratulations for your great unzip tool.
Is it possible to create a zip tool for MSX or it would take one day to zip a 16k file?

Regards
Marcos

Van Grauw

Ascended (8507)

afbeelding van Grauw

19-10-2015, 09:55

I don’t know how long compression would take but probably not a day Smile. I think it could be done at comparable speeds to lharc (which is pretty quick). There’s a trade-off between compression ratio and speed and memory consumption, which could be controlled by a command line option. Atm I don’t have plans to work on a that though...

Van cbsfox

Champion (327)

afbeelding van cbsfox

19-10-2015, 16:43

OK. But is it difficult to implement a zip tool?

Van Grauw

Ascended (8507)

afbeelding van Grauw

19-10-2015, 16:58

Assuming that a gzip compressor is a project comparable to gunzip;

Technically it’s quite challenging. Time-wise, looking at the gunzip commit history I worked on it for 14 days over the span of 6 months (excluding the test implementation in JavaScript).

Though now that I’ve implemented gunzip I could do gzip a little quicker, it’d probably still be a full holiday of work.

Van Parn

Champion (424)

afbeelding van Parn

19-10-2015, 17:10

This is awesome news and should be in the Spotlight. Why isn't it? Congrats to Grauw for his hard work and amazing achievement. Big smile

Van cbsfox

Champion (327)

afbeelding van cbsfox

19-10-2015, 20:11

Cool. Hope you can create a grip compressor as well.

Congrats
Marcos Daniel

Van Pippo

Champion (421)

afbeelding van Pippo

20-10-2015, 11:28

Hi Grauw,
you are a nice expert! Smile
This is my opinion.

Van Whizzy

Master (197)

afbeelding van Whizzy

20-10-2015, 22:15

Nice work Laurens ! very impressive

Van Pippo

Champion (421)

afbeelding van Pippo

21-10-2015, 07:48

Hi Whizzy,
I'm glad to meet you.
Please, can you reply to my post in software section?
Thank you.

Van jlemos

Resident (56)

afbeelding van jlemos

24-10-2015, 02:16

Very cool!

Thanks Grauw!!!!

Big smile