So any kind of probing for emulator extras would have to be feature based: instead of asking "what emulator are you?" the program would ask "what features do you have?". Also, this would not be limited to emulators: new hardware could respond to the same queries.
The software could ask for the emulator's name and version just to display it on the screen. Ok, a little pointless, because the user should know the emulator used. But it would look nice and somewhat ''proper''.
It would ask ''Do you have the required feature?''
If your software requires a certain feature, it would be sufficient to test if it is run on a certain version of a certain emulator. See the LaTeX mechanism for version checking.
I don't know how LaTeX checks versions. However, LaTeX is a single program, so you can make a perfect mapping from a version number to a feature set. There are multiple MSX emulators and new ones might appear in the future. You could make a mapping from (emulator name, emulator version) to a feature set at the time your MSX program is written, but that mapping will be outdated with future emulator releases.
LaTeX is a macro package for TeX. I'm not an expert in LaTeX and know almost nothing about the version checking in LaTeX. But there are lot's of packages to be included and I remember that there is an elaborated mechanism to ensure that you get the required version of a package. I think we could do something similar. The problem behind is much similar to getting the right virtual device implementation for an emulator, as I pointed out in my last post.
This would solve all the problems you mention according to your exemple.
Why do you need every software to run on the real thing?
Because if you drop that requirement, an emulator becomes a very awkward API for making PC games. Why limit yourself to 8-bit registers, small monochrome sprites, simple sound chips etc if you don't care about compatibility with the real hardware?
So while I think it is useful to add some enhancements for emulators to new MSX software, it should be optional features in my opinion.
Seems to be a very weird point of view. I gave an exemple why I e.g. like to have external filesystem access. I don't see it form a gamer's or game developer's point of view.
This thread originally started with a suggestion to make the MSX1 palette settable using an emulator enhancement, that's why I was thinking of games mostly. For development tools, I don't mind so much if they become emulator specific. We already have for example the debug device in openMSX, which is useful for logging during development.
Still, if it's possible to implement things in a way that is compatible with hardware, that is my preference. For example file system access should be done via BDOS calls, that way tools can access an external file system in an emulator but the same tools will still operate on MSX hardware.
You never know what the future brings. (Did we expect things like the OCM in 1985?) Even if a new Z80 with expanded instruction set is unlikely (really?), we should not mix emulator stuff with hardware opcodes. This seems elegant on the first gaze, but messes things up. Even amateur cores need not be aware of all emulator used opcodes. In the worst case, there must be another amateur cores adapted for every emulator for every HC system. Using opcodes here would cause problems on the hardware design level, which is even worse
What I mean is, why would a company have the right to add to an instruction set and a community would not? Unless the company is Zilog itself, of course.
Because the company can produce the hardware and the community normally can not. There are Z80 clones with additional opcodes, but these are illegal, aren't they? Of course, you can add to the instruction set, creating a variant of the Z80 or R800. Variants are bad here! But that's not my concern. I don't like to have emulator dependent stuff mixed up with hardware stuff if it could be avoided. This will most likely cause problems in the future, because you can never know what future brings. Call it experience or intuition.
The emulator might pass the feature/device request done via that ports to DLLs implementing the virtual device. This might also solve the problem with the feature to device mapping. This would be like the plug in of real devices in a real MSX. So one and the same emulator could be configured with different virtual devices at different times. Also updated virtual devices would be possible. The user would be responsible to provide that DLLs or the emulator itself could contact an online repository for devices and see if it can get something suitable. But then, you need to provide an unique name -- and version number -- for the virtual device. The software need to issue a device request to the emulator, and gets an errorcode or a device number for virtual device accessing purposes in return.
What do you mean with "DLL" exactly?
If it's a 32-bit Windows DLL, then it won't work on other platforms, which is a big deal for openMSX.
If it's any kind of loadable module containing native code, then it becomes a big effort to make sure it is available for all operating systems and CPUs. Or if it isn't available for all, some users will be missing out. Would the same binaries work in all emulators, or would a different version be needed for every emulator? The former is very difficult to design, the latter means yet more diversity.
If it's not native code but interpreted code, we must get agreement from all participating emulator authors on which interpreter to use and which functions it can call. Not an easy task.
So I think this approach is not feasible. Instead, every emulator should include its own implementation of every enhancement feature it supports. This is also how it is done for the traditional MSX devices: we share source code sometimes, but no-one would attempt for example an SCC implementation that is binary compatible between openMSX and blueMSX.
There is another problem: how is the information passed by that command? You might pass numbers (maybe randomly assigned to system clock frequencies) or (equidistant scaled) percent values. The former will be more hardware-like, the latter more approbriate to an emulator. A future hardware developer might have reasons to do that different that you expect it now.
I would prefer to use a setting that mimics the way hardware would do it. This makes it possible to later implement it in real hardware. It also avoids the spec favoring a particular emulator: although many parts of MSX emulators are very similar, there are differences in their designs as well.
Yes, I understand! But with that emulator extras I don't aim for later implementing it in real hardware. If you want to do it, that would be another issue. First do it with the emulator in a proper way, gaining experience for further expansions, because realizing it in hardware is far away, I think. (I also consider that whole emulator extras as gaining experience for maybe further MSX hardware, maybe MSX3. Therefore, we should not fix things unless we are pretty sure that they are useful.) If it's just changing emulation speed/changing chock speed, than all emulator implementing groups should agree in doing it in the same way. This could also hold for other emulation related issues. We are a community, aren't we?
I would prefer to use a setting that mimics the way hardware would do it. This makes it possible to later implement it in real hardware. It also avoids the spec favoring a particular emulator: although many parts of MSX emulators are very similar, there are differences in their designs as well.
Admittedly, that seems elegant, but I think that we should not mess up emulator stuff with real thingy stuff. You might want to just increase emulation speed for development, but keeping the speed of the emulated program original.
There are two different ways of speeding up in an emulator: increasing the CPU clock and changing the ratio between emulated time and real time. I think the former is more useful for games, since sound and interrupt timing would remain unchanged. The latter can be used for example to speed up an assembler, but in that case increasing the CPU clock would also be an option and would be more efficient as well (no need to emulate the VDP at a higher frequency).
The only reasons I see for speeding up by changing the emulated/real time ratio are:
- the MSX software might not behave correctly at a different Z80 clock
- the MSX software is slowed down by something other than the Z80
The former should not be the case for programs written to specifically support speedup. The latter is typically slow unskipable animations and it makes no sense for a program to work around its own bad design In some cases it might be VDP command engine slowness, but then a better solution would be to make a VDP extension to speed up the command engine.
So I think that for speed control by the MSX program itself, it only makes sense to manipulate the CPU clock. For speed control by the user, changing emu/real time ratio is very useful though.
I'm not an expert in the implementation of emulators.
What I mean is, why would a company have the right to add to an instruction set and a community would not? Unless the company is Zilog itself, of course.
Because the company can produce the hardware and the community normally can not.
With FPGA, the community can.
There are Z80 clones with additional opcodes, but these are illegal, aren't they? Of course, you can add to the instruction set, creating a variant of the Z80 or R800. Variants are bad here!
Incompatible variants are bad. Variants that add useful features in a compatible way are good.
By the way, I do agree that CPU clock setting should be done with I/O ports instead of opcodes, since this is easier for hardware implementations. I only think that there could be a good reason to introduce new opcodes at some point, even if there is no need for that right now.
But that's not my concern. I don't like to have emulator dependent stuff mixed up with hardware stuff if it could be avoided. This will most likely cause problems in the future, because you can never know what future brings. Call it experience or intuition.
The thing is that for most examples I've heard so far, they are not actually emulator specific. They are things that could be implemented in hardware in theory, but will be implemented in emulators first.
This thread originally started with a suggestion to make the MSX1 palette settable using an emulator enhancement, that's why I was thinking of games mostly. For development tools, I don't mind so much if they become emulator specific. We already have for example the debug device in openMSX, which is useful for logging during development.
Still, if it's possible to implement things in a way that is compatible with hardware, that is my preference. For example file system access should be done via BDOS calls, that way tools can access an external file system in an emulator but the same tools will still operate on MSX hardware.
Mixing hardware and emulator-dependent stuff would affect the compatibility with hardware!
Of course, file system access should be done via BDOS calls, but what I want is EXTERNAL file system access, that means access to the filesystem of the host computer of the emulator. Can this be done without modifying the BDOS or BIOS? If so, implementing that feature would be easy without accessing the emulator via a specific port directly. As I first had that idea, I thought it would be possible. I don't have deep knowledge of the internas of BDOS or BIOS.
What do you mean with "DLL" exactly?
If it's a 32-bit Windows DLL, then it won't work on other platforms, which is a big deal for openMSX.
As I work with Windows, it's a Windows DLL, but, of course, it would be better if it would be some more generic.
If it's any kind of loadable module containing native code, then it becomes a big effort to make sure it is available for all operating systems and CPUs. Or if it isn't available for all, some users will be missing out. Would the same binaries work in all emulators, or would a different version be needed for every emulator? The former is very difficult to design, the latter means yet more diversity.
It doesn't need to contain native code, but pseudocode where the implementating group of an emulator is responsible to provide the mapping to the respective platforms. But maybe we got to far for now. There may be several ways to go, of course.
If it's not native code but interpreted code, we must get agreement from all participating emulator authors on which interpreter to use and which functions it can call. Not an easy task.
Not an easy task, but feasable. There aren't so much emulators around. And if someone won't participate, ok, his decision!
So I think this approach is not feasible. Instead, every emulator should include its own implementation of every enhancement feature it supports. This is also how it is done for the traditional MSX devices: we share source code sometimes, but no-one would attempt for example an SCC implementation that is binary compatible between openMSX and blueMSX.
I already mentioned that it's an experimental thingy. We are a community and we can talk about what we're going to do. We can try several approaches. Would be very interesting and cool, wouldn't it?
So I think that for speed control by the MSX program itself, it only makes sense to manipulate the CPU clock. For speed control by the user, changing emu/real time ratio is very useful though.
I'm not an expert in the implementation of emulators.
In openMSX, CPU clock speed is selected by changing the "z80_freq" setting and setting "z80_freq_locked" to "false".
In openMSX, emu/real time ratio is changed by setting "speed" (default: 100%) or by disabling/enabling throttle (F9). Disabling throttle is basically the same as setting speed to infinity (meaning, as fast as possible) and setting frameskip to maximum (only 1 in N frames are actually drawn to the PC screen).
The difference is clear if the MSX is playing some music: changing CPU clock will have no effect since music replayers are timed on the VDP interrupt which will still be at 50/60Hz, while changing emu/real time ratio will change the speed at which the music is played, for example "set speed 150" will make it play 50% faster.