New wiki page: Emulation related file formats

Page 1/4
| 2 | 3 | 4

By TomH

Champion (327)

TomH's picture

21-12-2017, 16:29

I noticed a dangling link from Description of Common File Formats and therefore put up everything I have yet learnt about Emulation related file formats.

It's probably worth others reviewing and adding to it, as I doubt that my knowledge is complete. But I think it'd be really helpful to have that page populated, as the technical information is otherwise a little diverse and occasionally non-obvious.

Login or register to post comments

By Manuel

Ascended (18387)

Manuel's picture

22-12-2017, 10:45

Please use any text or information from this, if it helps: http://www.faq.msxnet.org/suffix.html (for any Wiki page of this site).

By TomH

Champion (327)

TomH's picture

22-12-2017, 20:32

Cool! Thanks for the tip, I've started adding to the wiki those file formats which I think pass the fairly informal 'emulation related' test that they're intended to represent an original piece of media.

You could probably get sucked into the semantics of what is and isn't 'emulation related' for an indefinite period given that both openMSX and blueMSX can mount a local folder (so .COM is emulation related, right? If I can use it from an emulator?) and that there are on-MSX options for mounting a disk image, using a virtual tape, etc.

By Manuel

Ascended (18387)

Manuel's picture

23-12-2017, 12:45

I would suggest to only include file formats that you would normally not encounter when using a real MSX. But nowadays that is also a grey area.

By Manuel

Ascended (18387)

Manuel's picture

28-12-2017, 21:59

On the DSK files you write that they need to have a file attribute table, but I don't think that's true. There is no content requirement at all, the format just describes the raw sectors. It's the same as you get when you copy a floppy disk device node to a file on UNIX: cat /dev/fd0 > my.dsk

By TomH

Champion (327)

TomH's picture

02-01-2018, 17:51

So as not to update with further questionable content: surely it's true that if there is no FAT, then the geometry of a DSK is unknowable?

Related question: does anybody ever circulate abridged DSK files? On some of the other platforms I'm aware of, such as the BBC Micro and sometimes the Sam Coupe, it is the convention that similar no-metadata raw sector dump files need contain only the sectors that have useful content. So if you freshly format a disk, the resulting disk image is as small as just the empty directory. If you're not using the whole disk and fragmentation is low enough that you have nothing relevant beyond, say, sector 12 then the disk image runs only up to the end of sector 12.

If not then presumably you can guess geometry based on image size?

EDIT: for the record, I'm in progress with an emulator of my own primarily for the benefit of myself, so I'm eating my own dogfood on anything I write on that page. Collecting the information will be the bigger service to the community, so I'd really like to get that page to be as accurate as possible.

I was also going to throw in a quick summary of the openMSX strategy for MegaROM detection, as both fMSX and WebMSX seem just to use SHA-based lookup tables but openMSX seemingly stopped doing that somewhere around a decade ago, and you've offered it as a model answer in the past but can you comment on why the ASCII 8kb mapper always has a point deducted at the end of voting? I assume it's just a necessary empirical shaping?

By Manuel

Ascended (18387)

Manuel's picture

02-01-2018, 17:53

The disk is a sector dump (like the UNIX command), it's not looked at the contents at all. Whether or not there are files on it or just raw data, is totally irrelevant. The disk is dumped as raw data in the order as you described.
This is also necessary, as some game disks do not use FAT, only a custom boot sector that loads the main program from disk using raw sector read commands. That main program loads also data via raw sector read commands.

The size tells a lot, but not all.
Size of 720kB must be a double sided double density disk. But a 360kB image can be either double sided single density or single sided double density. You can't tell. A 180kB image is probably single sided, single density. To make things simpler, on MSX single density is not used, so that helps Smile
Then there's also special things like extra sectors or extra tracks. An extra track (e.g. 81 tracks) fits in the format, but it does make the total size different from 360kB or 720kB.

Clear now? Smile

By Manuel

Ascended (18387)

Manuel's picture

02-01-2018, 17:57

About the mapper stuff: openMSX was actually the first to use a checksum-based method (like a database) to look up mapper types. But when a game isn't present in this database, it will use the good old guessing strategy you referred to. I don't know why 1 is subtracted. That code is very old. So, it's a good question Smile I'll ask around. Or join us on #openMSX on the FreeNode IRC channel.

By TomH

Champion (327)

TomH's picture

02-01-2018, 18:51

Oh, then my chronology is way off. Thank goodness I'm not attempting to document that!

Re: disk images; on various other platforms the dumping tools have the ability to inspect a sector dump and truncate it if it has a standard catalogue format and then demonstrably uses less than the entire surface of the disk. The emulator can fill in whatever it wants for the unrecorded area. Of course, that assumes the emulator knows the floppy geometry.

I'm aware that only a very confused emulator would try to factor filesystems into runtime disk image handling. I can't imagine a scenario in which it would be a good idea: the path of least resistance for '90s style hack-it-together emulation is some sort of sector map, and an emulator that wants accurately to support something anything restrictive than a DSK will obviously need to be at a higher level of intelligence than that.

I'll update the page when an opportunity arises.

By Manuel

Ascended (18387)

Manuel's picture

02-01-2018, 23:28

I'm not sure what exactly you mean with that 'confused emulator' paragraph. But the dir-as-disk feature sounds like something that fits your description.... Smile

By TomH

Champion (327)

TomH's picture

02-01-2018, 23:50

I dropped a word in the edit. Read as 'confused emulator author'. Ditto that 'support something anything restrictive' should have become 'support something anything less restrictive'. I'd adjust if there weren't a time limit on edits.

And I definitely didn't mean to include dir-as-disk: I'm not giving you a disk image, so you're not handling a disk image. Maybe you've implemented internally to look like a real disk to the disk controller, maybe you haven't, as a user I'm not really bothered either way.

I meant to distinguish only between emulators that can emulate real hardware and those that cannot. Unless you decided that dir-on-disk was a replacement for real media emulation, it's a distinct thing.

Page 1/4
| 2 | 3 | 4