Gunzip for MSX

페이지 4/11
1 | 2 | 3 | | 5 | 6 | 7 | 8 | 9

By Grauw

Ascended (8455)

Grauw의 아바타

21-10-2015, 01:05

Grauw wrote:
Louthrax wrote:

Ah, that option does not appear in 7zip GUI. Does gunzip already support that (uncompressed) format ?

Yeah, it’s the first format I added support for. It’s tested by the test file in res/test00.gz.

See Inflate_InflateUncompressed. It’s not very high performance (a block copy would be much faster), I didn’t bother optimising it as it’s rarely used, but if you want I can do it, it should be easy using the Writer_WriteBlock_IY method I added earlier today. Though it’s probably better if you support STORE as the gzip uncompressed data option is not commonly offered in the tool options it seems.

By the way, memory is pretty tight, the code is around 12K and then there’s a 4K read buffer and a 32K write buffer (which can’t be decreased in size), followed by a 5K heap… The stack is not far above that, so I use nearly all of the TPA. You might run into troubles with that.

Eventually for VGMPlay I will add memory mapper support so the write buffer can be two mapped 16K pages which would give some breathing room, but of course that means support for most MSX1 systems goes out the window. (For gunzip itself, I do want to keep supporting MSX1 and add DOS1 support.)

You can contact me by email if you need any assistance with that or other topics.

p.s. I’m going on a trip thursday so I might slow to respond until next week (not bringing a laptop).

By Louthrax

Prophet (2082)

Louthrax의 아바타

21-10-2015, 01:12

Grauw wrote:

By the way, memory is pretty tight, the code is around 12K and then there’s a 4K read buffer and a 32K write buffer (which can’t be decreased in size), followed by a 5K heap… The stack is not far above that, so I use nearly all of the TPA. You might run into troubles with that.

Argl, that was my main fear. The code to handle the zip structure, sub directories, etc... will take at least 7KB I think. I won't reuse all of your code of course (only inflate and dependencies), and heap management will be common-code, but that will definitively be tight !

By Grauw

Ascended (8455)

Grauw의 아바타

21-10-2015, 01:54

The classes you need to care about Inflate, Alphabet, Branch, FixedAlphabets, DynamicAlphabets, Reader, Writer, FileReader, FileWriter, NullWriter and crctable.asm. Those currently add up to a little under 8K. The rest can be either removed, slimmed down or replaced.

And the read buffer can be reduced in size, although disk I/O will be a less efficient. Finally, I could identify parts of the code to be more space efficient, at the cost of decompression efficiency.

By ricbit

Champion (438)

ricbit의 아바타

21-10-2015, 05:00

Grauw wrote:

32K write buffer (which can’t be decreased in size)

Why? Is it the size of the lz sliding window?

By ARTRAG

Enlighted (6249)

ARTRAG의 아바타

21-10-2015, 07:43

Using vram ?

By Grauw

Ascended (8455)

Grauw의 아바타

21-10-2015, 10:20

ricbit wrote:
Grauw wrote:

32K write buffer (which can’t be decreased in size)

Why? Is it the size of the lz sliding window?

Yep. Deflate’s dictionary size is 32K, which means to generate new bytes it can copy bytes from up to 32K ago.

ARTRAG wrote:

Using vram ?

It’s an idea, but on MSX1 you’ve got just 16K of it, and there’s overhead in accessing the VRAM as well.

By ricbit

Champion (438)

ricbit의 아바타

21-10-2015, 13:09

ARTRAG wrote:

Using vram ?

VRAM might be the best idea. Since it has an auto-incrementing pointer, you reduce register pressure on the main loop and free 4kb on main ram. There is an extra step of copying from disk to vram but I bet it is an overall win.

By Grauw

Ascended (8455)

Grauw의 아바타

21-10-2015, 13:44

Ah, you mean for the read buffer. Well, that might be an option, but:

1. How does the data get to the VRAM efficiently? Maybe via a 4K buffer in memory? Smile
2. I still need to do bounds checks as VRAM is not infinite, so probably the auto-incrementing won’t help.
3. TurboR puts a heavy penalty on VDP I/O.

By bore

Expert (115)

bore의 아바타

21-10-2015, 23:47

If memory is that tight, wouldn't it be better to just have a configurable read buffer size?
I guess that if you want to be able to use more RAM then you probably want to be able to use the VRAM too.

By Grauw

Ascended (8455)

Grauw의 아바타

22-10-2015, 17:09

Read buffer size is configurable, but if the block size is too small reading won't be very efficient.

페이지 4/11
1 | 2 | 3 | | 5 | 6 | 7 | 8 | 9