Emulation related file formats
This page was last modified 22:26, 11 January 2018 by TomH. Based on work by Mars2000you.

Contents

Cartridges

.ROM

ROM files are a raw byte capture of the data contained in a cartridge — a 128kb cartridge will produce a 128kb cartridge image.

They do not contain information about the hardware contained on the cartridge, if any: its paging mechanism, additional sound chips or any other feature. Since many of these images were generated on real MSX hardware rather than by pulling ROM chips, some 32kb images have their 16kb banks transposed as an effect of the means of capture.

An emulator must therefore adduce a lot of extra information in order properly to handle a ROM file; common solutions include ahead-of-time heuristics, cartridge image databases and dynamic machine reconfiguration at runtime.

Cassettes

.CAS

A CAS file is a capture of the decoded bytes stored on an MSX cassette in BIOS format, which uses a defined byte sequence to describe a header (i.e. a period of constant tone) of 0x1f, 0xa6, 0xde, 0xba, 0xcc, 0x13, 0x7d, 0x74. That sequence produces a header only if found within the file on an eight-byte boundary (i.e. starting at offset 0, or at offset 8, or at offset 16...).

A CAS file does not record:

  • the timing or spacing of stored data, including its baud rate and any gaps on the tape;
  • the difference between a long and short header.

A CAS file cannot contain:

  • data in any encoding other than that used by the MSX BIOS;
  • any file that includes the header byte sequence 0x1f ... 0x74 on an eight-byte boundary;
  • any file that is not a multiple of eight bytes in length other than as its final file.

It was originally designed to allow primitive emulators to implement a BIOS hack for tape loading and saving rather than emulating real hardware.

Some tools attempt to reintroduce long and short headers, and gaps, by guesswork based on file contents — if the header sequence is followed by ten bytes of the same value and those bytes are of value 0xd0, 0xd3 or 0xea then output the header as a gap followed by a long header. Otherwise output a short header, with no preceding gap.

.TSX

TSX is an extension of TZX[1], which increases the version number to 1.21 and adds a new block, Block 4b[2] to hold Kansas City Standard (i.e. the standard used by the BIOS) data compactly.

TZX is a heavyweight file format with a substantial amount of unnecessary complexity and redundancy but is capable of compactly describing an audio cassette at a very high resolution such that any original MSX tape can be preserved exactly.

.WAV

WAV[3] is a sprawling audio reproduction file format capable of containing raw or compressed audio data, using various different algorithms, at various resolutions and precisions.

It can exactly capture an original tape at a greater precision than a real cassette is capable of reliably maintaining.

As implemented by openMSX, both raw 8- and 16-bit WAV files, and those encoded with the MS-ADPCM compression scheme are supported.

Disks

All MSX disks are encoded with an MFM ('double density') bit stream; the WD2973 used in all MSX1, almost all MSX2 and Sony/Ciel MSX2+ disk interfaces uses a dedicated input pin to select between FM ('single density') and MFM encoding and are conventionally wired so that only MFM mode is available.

The TC8566AF (used in Turbo R, Panasonic/Sanyo MSX2+ and Panasonic MSX2 computers) is a completely different controller which implements the same command set Intel 8272 or NEC 765, as used in the IBM PC, rather than being a member of the WD family. FM or MFM selection occurs within the byte stream so those machines can read and write FM content but no known examples of such disks exist.

In contemporaneous MSX software the terms 'single density' and 'double density' are sometimes used with a less-common meaning of track density: single density is 40 tracks, double density is 80 tracks.

.DSK

A DSK is a raw sector dump of an MSX-DOS format floppy.

Sectors are ordered in the file:

  • by sector number as a primary key;
  • by side as a secondary key; and
  • by track number as a tertiary key.

So, for an 80-track double sided disk they are ordered as:

  • all nine sectors from side 1, track 1; then
  • all nine sectors from side 2, track 1; then
  • all nine sectors from side 1, track 2; then
  • all nine sectors from side 2, track 2;
  • etc.

DSK files contain no information about the disk geometry.

MSX floppies may be single or double sided but will usually be around 80 tracks — almost always 80 or 81. Emulators can therefore guess disk geometry based on file size: a 720kb image is double sided, a 360kb image is single sided.

Just as on the IBM PC, the first sector on a track is numbered as sector 1. So the nine sectors on each track are numbered 1 to 10. The first track is numbered as track 0. So if there are 40 tracks then they are numbered 0 to 39.

DSK files cannot properly describe:

  • any disk that does not conform exactly to the FAT standard of sectoring (i.e. size, IDs and relative placement) across its entire surface.

They therefore cannot capture most copy protection schemes, native CP/M floppies or any other kind of disk.

.MSX

MSX files are very rare and are exactly equivalent to DSK files in size, fidelity and exposition. However they differ from DSKs in track arrangement, arranging as:

  • sector number as a primary key;
  • track as a secondary key; and
  • side as a tertiary key.

So, for an 80-track double sided disk they are ordered as:

  • all nine sectors from side 1, track 1; then
  • all nine sectors from side 1, track 2; then
  • ...
  • all nine sectors from side 1, track 80; then
  • all nine sectors from side 2, track 1;
  • etc.

.IMG

An IMG file is a DSK file with a single additional byte providing drive geometry.

The first byte in the file can be:

  • 1, to indicate a single-sided 360kb disk image; or
  • 2, to indicate a double-sided 720kb disk image.

That byte is followed by a dump of sector contents in the same order as DSK.

.DMK

A DMK[4] file is a sampling of the decoded byte stream stored on a floppy, with additional metadata to identify FM and MFM ID address marks.

It does not specify the location of index, data or deleted data marks. Those must be guessed at based on the location of the ID address marks, according to that author's own thinking, with the consequence that not all data patterns that could reside outside of a sector can be stored reliably and different tools will interpret the same file differently.

DMK can capture all known MSX copy protection schemes, but:

  • fails to record a variety of syncs other than the ID address mark, as described above — and conflates MFM and FM marks;
  • cannot contain any sort of irregularly or weakly clocked data, such as fuzzy or weak bits, or missing clock bits other than the special case of an ID address mark;
  • cannot retain patches of data with different velocities;
  • cannot model write data splices, or any other event that would cause an ID address mark not to be on an exact byte boundary; and
  • offers no context as to whether the bytes between sectors should be encoded as FM or MFM.

The formal documentation does not define the order of tracks stored within the file, but it is conventionally taken to be interleaved — tracks are stored in the same order as a .DSK. Example pseudocode for reconstructing the underlying flux pattern from a DMK for a track is:

let IDAM offset list = the next 128 bytes of the file, 64 little endian two-byte values

while(track data remains, given track length declared in header):
  get distance from current track position to either:
      (i) the next IDAM; or
      (ii) the end of the track, if all IDAMs have been written.

  read that many bytes from the file and encoded as MFM.

  if there wasn't a next IDAM, stop. Track is complete.

  check the type of the IDAM: MFM or FM:
      (i) If it is FM, output the FM ID address mark; or
      (ii) if it is MFM, output the MFM sync word three times then the MFM-encoded byte 0xfe.

  swallow the 0xfe that is actually in the file.

  read and output the next six bytes as FM or MFM as per the IDAM mark.
  Keep a copy of the ID's fourth byte, to determine sector length.

  search for the probable start of sector: the next 0xfb or 0xf8 you encounter in the byte stream.
  To be really thorough, check against the IDAM table to make sure you aren't dealing with a
  disk that has an ID address mark but no subsequent sector data.

  output all the bytes up to that location. Swallow that byte. Then:
      (i) if the preceding IDAM was FM, output the found byte with clock 1100 0111;
      (ii) if the preceding IDAM was MFM, output the sync word three times then the found byte.

  output the next 2 + (length of sector) bytes in FM or MFM

Minor added complexities are that a file can be marked in its header as single-density (i.e. FM) only, in which case how a tool should deal with the presence of an MFM mark is undefined. If the file is not marked as single-density only then all single-density data on the disk is stored doubled up: if an FM sector contains the bytes 0x12, 0x23, 0x34 then the file will contain 0x12, 0x12, 0x23, 0x23, 0x34, 0x34.