Reading a file into "non-RAM" slots with MSX-DOS 2 functions

Page 1/2
| 2

By Louthrax

Paragon (1921)

Louthrax's picture

10-02-2019, 19:23

Hi all,

Does anybody know if there's a way to read a file into a non RAM slot using MSX-DOS 2 functions ?

I'd like to transfer data from a file directly into a Snatcher or SD-Snatcher cartridge (using no RAM buffer). These cartridges are not recognized as offical RAM slots by MSX-DOS2 at boot time (as they are by default in a non-writable state), but can be configured to be writable later.

I'm currently enabling write operations on the cartridge, setting the correct slots at 4000 or 8000, but the MSX-DOS 2 read function refuses to write in that SCC cartridge slot. I also tried to play with F342 and F343 addresses with no success.

Let me know if you have any hint on that. I'll use a RAM buffer for transfer (or try MSX-DOS 1 equivalent functions) if there's no fix for that, but that will complicate a bit my code.

Login or register to post comments

By Grauw

Enlighted (7794)

Grauw's picture

10-02-2019, 19:33

Afaik DOS2 can only load directly into the primary RAM mapper. If you want to load into a secondary mapper you need to use an intermediate buffer. I guess the same applies to non-mapped RAM? (Though I’m surprised a bit, but I guess it is the case.)

By Louthrax

Paragon (1921)

Louthrax's picture

10-02-2019, 20:07

Yeah, I read the documentation again (from your siteWink) :

Pages 0, 1 and 2 can contain any slot when doing a function call and will be preserved. Any parameters can be passed from the slot being selected, except that environment strings and disk transfer areas must be in the mapper RAM slot.

Any RAM segments can be selected in pages 0, 1 and 2 when an MSX-DOS function call or an MSX-DOS BIOS function call is made, and also the stack can be in any page. The current paging state will be preserved by all function calls even in error conditions. Any disk transfers will be done to the RAM segments which are paged in when the function call is made, even if they are not the original TPA segments.

Not sure though if "the mapper RAM slot" refers to "the primary RAM slot" or "any recognized mapped RAM slot". My guess is that MSX-DOS 2 uses segment-specific functions to ease the memory access in read-write operations, so it probaby just can't work on non-mapped slots (by design)...

Anyway, not a big deal, I'll switch to buffering.

By NYYRIKKI

Enlighted (5124)

NYYRIKKI's picture

10-02-2019, 20:23

BTW as it seems you are talking about Sofa-tools, I was wondering that could you maybe indicate more clearly on your www-page what tools support MSX-DOS2 and what tools require MSX-DOS2? Now I could not tell this from the descriptions.

By DarkSchneider

Paladin (769)

DarkSchneider's picture

14-02-2019, 09:50

MSX-DOS2 registers all the mappeable RAM. If it is not present on the table it generates after intialization, then we could say something like that RAM module is not MSX-DOS2 compatible.

Anyway, I recommend to use only the primary mapper, that will be the optimal RAM module (usually the larger one and the internal RAM in TurboR I think), and use only segments. DOS2 has many segment management functions, but in the case of slots, you have to manually switch, and that is much more dangerous. Also you have to make an "adaptable" code for any scenario, there are too many possible RAM modules configurations. Using only segments, you require a number of them, allocate them (and if not enough exit with error message), and go on. Also segments are very fast not like slots.

We can see that DOS2 was designed to not use much other RAM modules looking at how vaguely it handles it, after initialization, we get many info about primary mapper, and for the others we only get "entries", all within the same pocket:

Quote:

+8... Entries for other mapper slots. If there is none, +8 will be zero.

Then, if you ask me, I would do it reading into an intermediate buffer and then inter-segment write the data.

By Grauw

Enlighted (7794)

Grauw's picture

14-02-2019, 11:46

DarkSchneider wrote:

Anyway, I recommend to use only the primary mapper, that will be the optimal RAM module (usually the larger one and the internal RAM in TurboR I think), and use only segments.

No, no, I don’t recommend that at all! That goes completely counter to what I’m advocating. Software should prefer primary RAM generally, but it is very good practice to support non primary RAM as well. Otherwise you will get problems where people do have enough RAM plugged in in total, yet your software doesn’t work! Especially on the turboR ST this can be a problem, since the high speed internal RAM is always primary, but it is "only" 256K (with 64K taken away by DRAM mode and another 32K by DOS2).

Software using only the primary mapper for what is really just a tiny convenience is what causes people to feel the necessity and recommend others on this forum to do internal RAM upgrade modifications to their machine, even when there really is no need to do such because you can just plug in a mapper cart, which software should support (and luckily VGMPlay and several others do).

(That said, if your software only requires 128K of mapped memory, I think it is acceptable to only allocate from the primary mapper, since I can’t imagine a configuration where you would have enough total RAM and not have at least that much in the primary mapper. Nobody ever made 64K mapper cartridges, and the ST’s internal mapper is large enough as well.)

DarkSchneider wrote:

DOS2 has many segment management functions, but in the case of slots, you have to manually switch, and that is much more dangerous. Also you have to make an "adaptable" code for any scenario, there are too many possible RAM modules configurations. Using only segments, you require a number of them, allocate them (and if not enough exit with error message), and go on.

There’s nothing dangerous about switching slots. It’s a core mechanism of the MSX platform, software does it all the time, even more so than segment switching which overall is pretty rare (due to mapper usage being relatively uncommon comparatively).

And I don’t get your argument that you have to account for many scenarios with many RAM module configurations? The RAM is in a slot, the software doesn’t need to be different if it’s in one slot or another, just use it. The only gotcha is disk transfer, which is unfortunate (and I don’t know why), but if that’s the only thing you need to deal with it’s easily worked around.

DarkSchneider wrote:

Also segments are very fast not like slots.

You’re presuming you have to switch slots all the time, that is not the case, generally the slot switching can be omitted if the mapper segment is in the same slot. Only if it’s in a different slot do you need the slot switching overhead, however in that case a slight performance cost is still much better than the software not working at all due to it not being able to use all the available RAM.

Actually in VGMPlay I always switch slots, but it still only needs to happen every couple of seconds, and but if I would implement this check-before-switch optimisation it would be even more infrequent, only once every tens of seconds depending on the mapper sizes.

DarkSchneider wrote:

We can see that DOS2 was designed to not use much other RAM modules looking at how vaguely it handles it, after initialization, we get many info about primary mapper, and for the others we only get "entries", all within the same pocket:

Quote:

+8... Entries for other mapper slots. If there is none, +8 will be zero.

You’re misinterpreting that, it’s not undefined. The entries for the other mapper slots are exactly the same as for the first, 8 bytes for each mapper present in the system.

By Louthrax

Paragon (1921)

Louthrax's picture

14-02-2019, 11:46

NYYRIKKI wrote:

BTW as it seems you are talking about Sofa-tools, I was wondering that could you maybe indicate more clearly on your www-page what tools support MSX-DOS2 and what tools require MSX-DOS2? Now I could not tell this from the descriptions.

You're right, I'll add a clearer indictation on this. All tools are MSX-DOS 2 specific. Making crippled MSX-DOS 1 versions is doable, but I'm more thinking about a kind of MSXDOS1 to MSXDOS2 adapter TSR program to handle this cleanly.

By Grauw

Enlighted (7794)

Grauw's picture

14-02-2019, 11:51

Louthrax wrote:

All tools are MSX-DOS 2 specific. Making crippled MSX-DOS 1 versions is doable, but I'm more thinking about a kind of MSXDOS1 to MSXDOS2 adapter TSR program to handle this cleanly.

Adapting VGMPlay to MSX-DOS1 has also been requested in the past but is no longer something I plan to do, DOS2 is convenient to use and it’s too much hassle to have to deal with two environments. DOS2 is widespread enough nowadays I think. But a MU-like TSR which also adds a compatibility layer for the disk functions could be a cheap way for me to support it after all, so I like that concept.

By DarkSchneider

Paladin (769)

DarkSchneider's picture

14-02-2019, 12:01

With the main code and the main data in primary, plus the ISR behaviour...handling a 2-dimensional array of memory could reach to many unexpected crashes. The code complexity grows exponentially, and focusing on making use of those useless small RAM modules attached to the slots, takes precious resources from focusing on primary and its segments, getting all the power from what inter-segment access and calls can do with the proper use.

What is the limiter? If the worst case is the A1-ST, well it was designed to have 256KB. The TurboR we know is a bit special, from one side you can think it is limited to 256KB, from the other side you can use more than that, but accessing to external RAM kills the TurboR itself. So there is no good case really. As mentioned, the TurboR is special.
Any modern loader or device with RAM includes 512KB or at least 256KB (usually 512), more than enough, and focusing on using that primary, like loading and executing code on the fly, can get much more.

If someone wants to feel fulfilled using his old pieces of junk like those 16/32/64 KB RAM modules, then sorry that is not my target at all. Load old software for that old stuff.
Modern software modern methods, in this case a linear RAM segmentation handled by the OS. Working in this way is much more productive, checked by my own, so at least for me.

By konamiman

Paladin (981)

konamiman's picture

14-02-2019, 12:28

Louthrax wrote:

Not sure though if "the mapper RAM slot" refers to "the primary RAM slot" or "any recognized mapped RAM slot"

It must be the primary RAM slot, and this is documented somewhere, I don't remember exactly where. As a matter of fact the NestorBASIC functions to read and write files to RAM work on any mapper, primary or not; but that's because under the hood it's using buffering when the source/destination slot is not the primary one.

By Grauw

Enlighted (7794)

Grauw's picture

14-02-2019, 13:43

DarkSchneider wrote:

With the main code and the main data in primary, plus the ISR behaviour...

Banking can be annoying for ISRs because they need to make sure it can access the data that it needs (that’s why you generally want to keep all ISR-related stuff in page 3 which never changes). But the mechanism used to do banking (mapper, slot or mapper + slot) doesn’t make a difference in risk.

DarkSchneider wrote:

handling a 2-dimensional array of memory could reach to many unexpected crashes. The code complexity grows exponentially

Don’t think of it as a 2-dimensional array, the DOS2 mapper routines already mostly abstract this away from you, which is one of its good points. Rather think of it as each segment being identified by a 16-bit identifier, the low part returned from ALL_SEG in register A, and the high part in register B.

The only part where the mapper routines don’t abstract the slots away from you is that to select a slot, you just need to do one call instead of two. See this from the perspective of, it’s giving the software full control, precisely to make optimisations if they need to, like preferring the primary mapper for code.

I think saying that just this introduces "exponential complexity" is a hyperbole, it really all comes down to this (in my object-oriented style so don’t mind my use of IX here):

;
MapperSegment: MACRO
    segment:
        db 0
    slot:
        db 0
    ENDM

; ix = this
MapperSegment_SetPage2:
    ld a,(ix + MapperSegment.slot)
    ld h,80H
    call ENASLT
    ld a,(ix + MapperSegment.segment)
    jp PUT_P2

Or if you want to optimise it so ENASLT only gets called when necessary:

; ix = this
MapperSegment_SetPage2:
    ld a,(ix + MapperSegment.slot)
    ld hl,MapperSegment_currentSlotPage2
    cp (hl)
    ld (hl),a
    ld h,80H
    call nz,ENASLT
    ld a,(ix + MapperSegment.segment)
    jp PUT_P2

;
MapperSegment_currentSlotPage2:
    db 0
DarkSchneider wrote:

and focusing on making use of those useless small RAM modules attached to the slots, takes precious resources from focusing on primary and its segments, getting all the power from what inter-segment access and calls can do with the proper use.

Tell me what is the use case where you would need to handle so much bulk data that you actually would even use such amounts of RAM, while at the same time being so performance critical that a few extra cycles for slot switching would hurt you. I find it hard to imagine such a case, bulk data and random access are inherently at odds with critical performance.

And also consider that for users who would actually run into extra slot switching overhead, the alternative is to have the software not work at all. While it’s a small effort for the programmer to support it, just one ENASLT call as I showed above.

DarkSchneider wrote:

What is the limiter? If the worst case is the A1-ST, well it was designed to have 256KB.

Yes, and because some programmers reason like you do, now whenever we get an ST offered for sale we get stupid comments like "oh you should expand the internal memory, it’s useless like this" which is complete and utter BS. So that’s why I’m always so vehemently making the case for properly dealing with mappers in multiple slots in several threads here in the forum.

DarkSchneider wrote:

The TurboR we know is a bit special, from one side you can think it is limited to 256KB, from the other side you can use more than that, but accessing to external RAM kills the TurboR itself. So there is no good case really.

For performance, only the program code really needs to live in internal memory and for that 256K is plenty. VGMPlay barely slows down once the internal memory is depleted and it starts reading data from external memory.

DarkSchneider wrote:

If someone wants to feel fulfilled using his old pieces of junk like those 16/32/64 KB RAM modules, then sorry that is not my target at all. Load old software for that old stuff.

Of course, those are not the topic, we were talking about mapped memory extensions, and the smallest I’ve ever seen is 256K (externally).

DarkSchneider wrote:

Modern software modern methods, in this case a linear RAM segmentation handled by the OS. Working in this way is much more productive, checked by my own, so at least for me.

Yes, and my point is that it *is* handled and abstracted away by the OS. DOS2 keeps track internally of all the different memory slots and their allocation, all you need to do is call two methods to switch. A small effort.

Page 1/2
| 2
My MSX profile