SCC-I hidden potential unlocked?

Paladin (683)

22-09-2020, 20:27

Not much progress lately.
I did increase the range for pitch bend velocities, but had to change the format a bit so the previous songs are now broken (I'll fix it later).

Other than that I got a bit curious about some Konami compositions for the SCC and managed to convert a SGM file to a readable text file, so I could have a peak. I ended up typing it over and played around a bit with some "morphing" wave forms, not a lot so the original sound & instruments are still kept intact as much as possible, just to have a bit of comparison.
So, for those interested, I made small video with the same song played twice using plain "static" wave forms and "morphing" ones.

Battle Ship

Van JohnHassink

Ambassador (5665)

22-09-2020, 21:29

Huh wow, awesome recreation(s).
By the way, that tracker seems to load and decrunch its files pretty damn fast, considering all the music and wave data its format probably must store!

Van GhostwriterP

Paladin (683)

22-09-2020, 22:55

Wave data if only up to 16 wave forms per song, all manipulations and morphing is currently done on the fly using these 16 "basic" wave forms.

In truth there is not a lot extraction going on, since save format is nearly identical to the in memory format (also in a condensed/compact format but occupying more memory - if that makes any sense), but you can leave this up to R800.

Current file size is 10.3 kb for battle ship and biggest file so far is metal squad with 15.2 kb, so I don't think it is needed to add some additional compression on this.

edit: and song data is loaded into v9990 vram first then extracted for the whole song at once, so this speeds up the loading part.

Van niek

Expert (112)

30-10-2020, 22:34

If you can export the files to VGM those should be playable on a normal MSX right?

Van sdsnatcher73

Prophet (3849)

31-10-2020, 08:42

niek wrote:

If you can export the files to VGM those should be playable on a normal MSX right?

In theory this may work but as I understand the player updates the waveforms often and this requires a lot of CPU. You can try playing back Space Manbow VGM's on a MSX1/2/2+ and notice the songs do not entirely sound like in the game. This is because of the waveform updates as I understood from Grauw. It plays back nicely on a turbo R. Also in the RealFun 3 player the waveforms are transformed from a base waveform (well up to 16 base waveforms). VGM format just records the writes to the chip so a VGM file from a RealFun3 track will have to contain the 16 base waveforms but also ALL transformed versions. This will add up to a lot of different waveforms I guess. So it will be a bit inefficient (but with enough RAM, VGMPLAY should cope).

Van GhostwriterP

Paladin (683)

31-10-2020, 20:56

A while ago I had a look at solid snake opening VGM file, which is extracted 1.23 MB. It uses only about 83 distinct wave forms but it is repeated (duplicated) all the time in the VGM, which makes it really big.

In solid snake wave morphing is only used on two or three channels tops, so for 5 channel morphing a VGM file would possibly double in size, if not even more

Don't know it is possible to play the .VGZ directly with VGMPLAY, that one is only 109 kB.

Van Pencioner

Scribe (1546)

31-10-2020, 21:02

VGZ is being decompressed into memory before playback, so you need a big memory mapper anyway...

Van Grauw

Ascended (10707)

31-10-2020, 21:47

Pencioner wrote:

VGZ is being decompressed into memory before playback, so you need a big memory mapper anyway...

Yes VGMPlay supports VGZ.

sdsnatcher73 wrote:

You can try playing back Space Manbow VGM's on a MSX1/2/2+ and notice the songs do not entirely sound like in the game. This is because of the waveform updates as I understood from Grauw.

This is due to the generic nature of VGM and VGMPlay; I need to swap in the SCC for every waveform byte written, as I can’t keep the slot selected because other chips may need to be activated. Even though that slot select routine is very optimised, it is still a lot of slot selects.

VGM is a good intermediary format with all its player, tools, emulator and other software support. You can record the Realfun3 output to VGM using openMSX. However I always think for playback in games and such you’ll want to convert the VGM data to a custom format which optimises for MSX playback of specific chips; e.g. an SCC player should group all 32 bytes of a waveform change into a single command.

For a custom VGM-like stream-of-commands format, you can apply offline grammar-based compression. Basically, introduce “call” and “return” to the command stream and extract repeated sections. The benefit of this kind of compression compared to LZ77-based formats (like gz) is that you do not need a decompression buffer, you can play straight from the compressed data. I’m working on this at the moment for one of my players, the compressor is tricky to implement but the player is simple.

Van niek

Expert (112)

31-10-2020, 23:56

Well it will be interesting to see how well it actually works right? Afaik the player + music is not available as of yet, so if GhostwriterP could do some vgm_rec...?

Van Grauw

Ascended (10707)

01-11-2020, 01:26

The current player I’m implementing this for is for OPM/OPNA/OPNB and the other one that I want to implement it for is for OPLL+PSG, so either way it won’t be readily available for SCC. Though all code is open source of course, and I might be convinced in the future to make an SCC adaptation of it.

And I am using VGM as the source format, to allow complete freedom for how the music is composed. The conversion to a custom format is to improve the Z80 processing efficiency and reduce the file size, by having more compact representation for the commands and by allowing me to apply this compression.

The compression is already partially working by the way, when I only apply one level of compression (one section never calls another) I get a 38% reduction in the number of commands on the set of tracks I’m testing with. The recursive implementation that I’m currently working on will probably further reduce the size by about as much.