Synthesix

Page 4/8
1 | 2 | 3 | | 5 | 6 | 7 | 8

By Grauw

Enlighted (7791)

Grauw's picture

10-09-2014, 01:02

hit9918 wrote:

I think that as soon as actualy something is installed in fd9a, a callslt to a cartridge, that makes a too long DI zone.

I don’t think many cartridges do that though. But, we’ll see what pops up. Whatever it is something can be done about it I’m sure Smile. Some easy solutions: remove the offending cartridge, run at higher CPU clock frequency, or use a buffered MIDI interface like the bit² MIDI interface (not supported yet). More complicated solutions are for Synthesix to manually deal with any particular troublesome hardware.

hit9918 wrote:

p.s. is there a way to run synthesix on bluemsx?

In CocoaMSX, I simply select the ROM and select Konami SCC mapper type…

mars2000you wrote:

Don't use a TurboR machine, it will hang.

Odd, in my CocoaMSX FS-A1GT machine it works fine…

By mars2000you

Enlighted (5413)

mars2000you's picture

10-09-2014, 01:03

Didn't have the time to check several versions, you must read :

"Don't use a TurboR machine on recent betas, it will hang. But no any problem on the first 2.8.3 beta"

By hit9918

Prophet (2824)

hit9918's picture

10-09-2014, 01:31

Ah I always forget, select ROM type and it works.
I'm too working on an IM2 server for page 3 bios. On TurboR one needs to flip to R800 and set that one to IM2, too. Because it is one of those unsaveable states, so the cpu flip code can't see it.

By Grauw

Enlighted (7791)

Grauw's picture

12-11-2014, 11:59

Continuing the off-topic discussion from the PSG de-emulation thread here...

Quoting for reference:

Grauw wrote:
redman wrote:

I think that for something like Synthesix you should comletely forget about interrupts and scan midi & drive PSG as fast and often as you can. Maybe use interrupts only for arpeggio's and such. But that's just my opinion :)

Mmh… maybe, but then the update frequency becomes irregular. I think that’d be audible. Additional complication is that you can wire triggered modules to run at control frequency by just feeding them a constant 127. Is handy for ADSRs etc. In editing mode I need some time for the UI as well so there a fixed update frequency is better.

But as I was thinking about adding a 100Hz control rate mode anyway, maybe I should also just make an unlimited setting in recording mode. Then people can decide on their own which they prefer, depending on the patch and machine speed (turboR :)). I’ll just have to find a good way way to discourage people from using aforementioned constant trigger method.

redman wrote:
Quote:

Mmh… maybe, but then the update frequency becomes irregular. I think that’d be audible.

With something like midi you want to react as soon as possible and not wait untill the next interrupt to start the note. But then, if you generate an envelope or lfo or other internal control signal you want to have that in a regular stream.

Isn't there an interrupt for incomming midi? I mean, midi has a command speed of about 1kHz. That's much more than 50 or 60 Hz, right? How is it handled?

Quote:

In editing mode I need some time for the UI as well so there a fixed update frequency is better.

I can imagine two scenario's, both involving line interrupts.
It depends on if you get an interrupt from the midi in port or not.
If yes then handle that data immediately.
If no then bunch it together with the control generation.

The control signal generation could run on a line interrupt. If you divide the screen in, say, 8 parts, then you already have a rate of 400Hz, which is decent.
You'd get (i think?) about 9000 z80 cycles between line interrupts which seems pretty decent as well. Maybe you could even bump it up to 800 Hz by halving the line offset, which gets you close to midi command rate.
You may need to tune it to not get choked on incomming midi data. Actually, i think midi data is a concern all thoughout the path, whatever design you use. There is probably little buffering at the midi port so if you don't read it fast enough you risk losing data. So the faster you handle the midi data the better (up to about 3k midi bytes/s). How do you handle this problem, or, is this a problem at all?

Since this would all be running in the background on interrupts you can do your UI more or less the usual way as long as you don't make it too heavy.

Anyway, just theorizing :)

By Grauw

Enlighted (7791)

Grauw's picture

12-11-2014, 11:57

redman wrote:

With something like midi you want to react as soon as possible and not wait untill the next interrupt to start the note. But then, if you generate an envelope or lfo or other internal control signal you want to have that in a regular stream.

Synthesix’s core is using a “pull” architecture, where you basically tell every module to update and they just read the values from whatever source the inputs are referencing. This is efficient and practical, because I don’t need to deal with variable size update lists (one output can point to multiple inputs after all) or receiving update calls from multiple inputs (inefficient).

But it means I can’t propagate a single value change without updating all modules.

redman wrote:

Isn't there an interrupt for incomming midi? I mean, midi has a command speed of about 1kHz. That's much more than 50 or 60 Hz, right? How is it handled?

Interrupts indeed.

redman wrote:

I can imagine two scenario's, both involving line interrupts.
It depends on if you get an interrupt from the midi in port or not.
If yes then handle that data immediately.

If I would remove the rate limiting it would essentially do that, when it’s not still busy updating from the previous input. As long as things that need to run at a fixed rate (like ADSR) take their trigger input from a clock signal, their speed should stay constant. Although there will still be some irregularities in the exact timing.

Also consider, if there’s a note-off immediately followed by a note-on, it’d be inefficient if I would kick off the update after the note-off. So a bit of buffering won’t hurt.

redman wrote:

If no then bunch it together with the control generation.
The control signal generation could run on a line interrupt. If you divide the screen in, say, 8 parts, then you already have a rate of 400Hz, which is decent.

Note that at 60Hz, the screen displays 313 lines every frame. At 50Hz, 263 lines. There is a lot of border lines at the top and bottom. The number of display lines that you can have a line interrupt on is 212. So, if you want to have even distribution of intervals, at 60Hz the minimum interval is 101 lines (313 - 212), at 50Hz it is 51 lines (263 - 212). Because these need to stay aligned, the highest practical rates are 180Hz (every 104 lines @ 60Hz) or 250 Hz (every 53 lines @ 50Hz).

There’s one possible workaround: enabling overscan will probably allow you to have interrupts further down the border (up to 256 lines), but I’m sure it introduces additional complications, and things are starting to get pretty hairy at this point.

Note that MSX-AUDIO, OPL4 and most MIDI interfaces have a configurable timer interrupt. It’s probably a lot less trouble to just use these when available, and sticking to vsynced rate if not.

redman wrote:

You'd get (i think?) about 9000 z80 cycles between line interrupts which seems pretty decent as well.

The patch in the alpha release needs double that to compute a single update. On a Z80 it is consuming about 30% CPU. On the R800 less than 10%.

How much CPU is used exactly depends on the complexity of the patch, the polyphony, and the update frequency. Also it spikes a bit when there is a lot of input to be processed. I do have some further optimisations planned but at the same time I also expect patches that are more complex or use much more channels. For example, OPL4 has 18 FM channels as opposed to the SCC’s 5, although it does have hardware ADSR so the complexity will generally be lower.

redman wrote:

Actually, i think midi data is a concern all thoughout the path, whatever design you use. There is probably little buffering at the midi port so if you don't read it fast enough you risk losing data. So the faster you handle the midi data the better (up to about 3k midi bytes/s). How do you handle this problem, or, is this a problem at all?

Most MIDI interfaces don’t have any buffering at all. All MIDI data is received immediately and put into a 256-byte circular buffer. This buffer is then processed 60 times per second, after which the modules update.

By redman

Expert (67)

redman's picture

12-11-2014, 13:56

Quote:

Most MIDI interfaces don’t have any buffering at all. All MIDI data is received immediately and put into a 256-byte circular buffer.

Well, there is a one byte buffer in the interface Tongue
Ok, so it is interrupt driven, which is good!.
Also, the circular buffer is large enough to handle all incoming data even if processed at low rates (50Hz), which is also good.

Quote:

If I would remove the rate limiting it would essentially do that, when it’s not still busy updating from the previous input. As long as things that need to run at a fixed rate (like ADSR) take their trigger input from a clock signal, their speed should stay constant. Although there will still be some irregularities in the exact timing.

Yes, the actual timing would be quantised to the refresh rate of your engine.

Quote:

Also consider, if there’s a note-off immediately followed by a note-on, it’d be inefficient if I would kick off the update after the note-off. So a bit of buffering won’t hurt.

Well, my idea is that the midi data is pre-processed to a certain degree to change state in your control generator. The idea is that there may be a lot of important musical data in between that will require some special state the next time the engine updates.
If a note off happened, followed by a note on then that is important musical information. In that case i would expect the envelopes to retrigger. If there was no note off and only a note on i could expect the envelopes not to retrigger but instead activate a slide function or something like that.
This is, i guess, what i mean by processing midi data ASAP. You don't need to actually do anything like setting soundchip registers at this point, you just need to be aware of what happened and make sure the engine is aware of these changes and acts accordingly on it's next run through. Some data could be processed on a 'last seen value' base. So by processing the midi data you just remember the last value of that particular midi CC.
I think this is just as efficient as reading through the whole buffer once in a while with the advantage of spreading out this processing in time. You don't get a big fat 256 byte buffer to process all at the same time. And the processing of the midi data should not be as heavy as actually updating the engine while still extracting the necessary information for the engine to run correctly on a lower frequency.

Then the engine functions probably like it does now.
The suggestion there is to make it run at as high a (steady) frequency as possible.
Your remarks about the line interrupts and blanking period are very enlighting!
I thought that this may be a problem. But i also immediately thought that there must be some trick to disable the vblank period. Wasn't there a t rick where you could basically start a new display period halfway the screen? Or do screensplits work differently (just setting the mode, but not restaring the draw, i have no actual experience with this Smile )?
This does indeed take you into difficult places. For one, you probably will need to disable (parts of) the regular vblank interrupt routines and write your own to handle all the system stuff. And that update will be pretty heavy as well and it needs to fit, together with the midi handling and engine, within one period between lines.

A lot of stuff to think about for sure!
Smile
I think that i can't comment anymore without having a much deeper understanding of how you organize stuff.
You may ask yourself, why the hell did you start about it in the first place then?
Well, it's because YOU STOLE MY IDEA!!! Wink
Actually, a couple of years ago i started fantasizing about turning my msx into a midi module so i had a lot of thoughts about how it may work.
So it's realy nice to see someone actually do it!
And maybe now i feel way too much compelled to change YOUR idea into mine, which is not so nice.
I just want you to understand that you probably have way more experience with this particular set of problems than me so, at least, don't take everything i say TOO seriously.

Gotta run now!
greets!

By Grauw

Enlighted (7791)

Grauw's picture

12-11-2014, 23:03

redman wrote:

Well, my idea is that the midi data is pre-processed to a certain degree to change state in your control generator. The idea is that there may be a lot of important musical data in between that will require some special state the next time the engine updates.

Currently the control update routine (triggered by the 60Hz interrupt) consist of three steps:

  1. The MIDI data is processed, and the values of the channel and polyphonic note output modules are set. For example note value, gate, trigger, channel CC values, etc.
  2. All modules are updated. This is the part with the “pull” architecture, and includes updating the sound chips.
  3. All trigger values are reset so that for the next control update they are 0 again.

In the remaining time the editor and UI run on the main loop.

redman wrote:

This is, i guess, what i mean by processing midi data ASAP. You don't need to actually do anything like setting soundchip registers at this point, you just need to be aware of what happened and make sure the engine is aware of these changes and acts accordingly on it's next run through. Some data could be processed on a 'last seen value' base. So by processing the midi data you just remember the last value of that particular midi CC.

Ah, I see what you mean.

In editor mode MIDI data processing and module updating is done at the same time, as shown above. However I plan for the “recording mode” to do the data processing (step 1 above) continuously on the main loop so the latency is reduced. UI is disabled in this mode.

However, I have yet to see how much difference that will really make. It will reduce the peak latency for sure, but in the general case there won’t be too many messages to process. The update frequency is the primary source of latency, 60Hz means an average latency of 8 ms.

redman wrote:

I think this is just as efficient as reading through the whole buffer once in a while with the advantage of spreading out this processing in time.

If by spreading out processing over time you mean processing the MIDI data immediately on the interrupts where they are received, I don’t think this is possible, or practical. There’s only 1145 cycles available per byte, half that if you consider the case of two MIDI interfaces plugged in, and some handlers are already quite slow (e.g. Yamaha SFG MIDI requires a slot switch). Just push/popping four extra registers uses 112 cycles for every byte received. I’d have to be extremely careful to never exceed the timing budget, with some room to spare, whereas currently it all just kinda averages out.

But maybe more importantly, processing the MIDI data on the interrupt means that the state of the MIDI channel/note modules will change while others are updating based on their values. Although this might reduce latency a little in certain cases, it can cause weird behaviours. E.g. when I have a single note controlling two SCC channels, the first may get the old pitch while the second gets the new one.

And all this extra stuff happening on the MIDI interrupts also means less time remaining for the module update, delaying the completion of it, adding back the latency that you might have gained by avoiding the processing at the start. I think it’s better to do the processing of MIDI data during the idle time between modules updates.

redman wrote:

You don't get a big fat 256 byte buffer to process all at the same time.

By the way, the buffer is 256 bytes mostly because it’s faster to implement a circular buffer that way. Given the MIDI transfer speed, at most I receive 53 bytes per control update, and in the common case much less.

redman wrote:

Then the engine functions probably like it does now.
The suggestion there is to make it run at as high a (steady) frequency as possible.

Yeah, it’s making more and more sense to me to provide different settings for the control rate. Especially on the turboR there should be no problem increasing the frequency.

redman wrote:

Your remarks about the line interrupts and blanking period are very enlighting!
I thought that this may be a problem. But i also immediately thought that there must be some trick to disable the vblank period. Wasn't there a t rick where you could basically start a new display period halfway the screen? Or do screensplits work differently (just setting the mode, but not restaring the draw, i have no actual experience with this )?

The only trick I know of is the overscan mode. If you are in 212 line mode, and switch to 192 line mode between lines 192 and 212, the VDP will skip the vertical blanking and just keep on scanning. The line number for the line interrupt is still an 8-bit value though so 255 is the maximum. Also, the VDP is operating outside normal conditions and I know there is some weirdness displaying line -1 in text modes, so there might be additional issues with line ints as well. However it can probably be done.

redman wrote:

This does indeed take you into difficult places. For one, you probably will need to disable (parts of) the regular vblank interrupt routines and write your own to handle all the system stuff. And that update will be pretty heavy as well and it needs to fit, together with the midi handling and engine, within one period between lines.

I’ve already completely replaced the BIOS ISR in Synthesix Smile. The new ISR just initiates the control update “thread”.

The keyboard scanning happens as part of the MIDI processing (it’s considered a MIDI device). Actually, it’s quite costly, iirc it takes 10 scanlines (~2000 cycles) to complete even when no keys are pressed.

redman wrote:

I think that i can't comment anymore without having a much deeper understanding of how you organize stuff.
You may ask yourself, why the hell did you start about it in the first place then?
Well, it's because YOU STOLE MY IDEA!!! Wink
Actually, a couple of years ago i started fantasizing about turning my msx into a midi module so i had a lot of thoughts about how it may work.
So it's realy nice to see someone actually do it!
And maybe now i feel way too much compelled to change YOUR idea into mine, which is not so nice.
I just want you to understand that you probably have way more experience with this particular set of problems than me so, at least, don't take everything i say TOO seriously.

Haha, you’re welcome. It’s nice to discuss the project, and I got some more ideas on how to deal with this update frequency stuff.

I was beaten to it too, by the way: MsxKun made his MSX Synth software. I cursed a bit when I found out after I had started Synthesix development hehe Smile. But I think we have quite different approaches, so they are complementary products.

I have to say the scope of this project is much bigger than I imagined. The list of ideas is nearly endless :). If you’re interested in taking a peek at the source code, it’s available.

By hit9918

Prophet (2824)

hit9918's picture

13-11-2014, 09:14

A code for a fast interrupt handler filling a ringbuffer.
The deal is that the consumer code outside the interrupt can afford much more logic and can have much more hiccup in time.

chainnext: 	jp ...filled in by p3bios...
chain:		in a,(devicestatus) : rlca : jr nc,chainnext	;was not my device, not handeled
		in a,(devicedata)
ptr:		ld (0),a
		ld a,(ptr+1)			;+1 skip opcode byte
		inc a				;index in ringbuffer
marker:		cp 0 : jr z,ovl
		ld (ptr+1),a
		xor a				;flag info for p3bios about return mode
		ret
ovl:		ld a,error
		ld (overflow),a
		xor a
		ret
overflow:	db 0

Those "ld (0),a" and "cp 0" are about selfmodifying code. Faster than push / pop hl, ld hl,() ...
It is assumed that bios did push AF and restores it and does EI.

The dumb interrupt blindly fills a 256 bytes ringbuffer. It feels a bit like DMA hardware Smile
The consumer is responsible for all the setup of "ptr" and "marker" and properly consume from the ringbuffer.

About line interrupts, grauw you mixed the pal vs ntsc line numbers. It is pal who has the 312 lines.
Here a sheme of 150Hz. Well it's not a high rate but same rate on PAL and NTSC.

		which interrupt distance to chose.
		312 / 6 = 52
		260 / 5 = 52
		both are multiples of 52
		pick 104 lines interrupt distance		

		
		PAL: 312 / 104 = 3 interrupts
		line 0
		line 104
		line 208


		NTSC: one frame with 3 interrupts, one with 2 interrupts:
		3 interrupts:
		line 0
		line 104
		line 208
		
		2 interrupts:
		208 + 104 = 312. modulo 260 lines of NTSC frame, 312 - 260 = 52
		-> line 52
		52 + 104 = 156
		-> line 156

A question is whether one can get more with opened borders.
One day I tried an int below vblank line and got inconistent things with bluemsx vs openmsx.
Someone with real machine would have to check out.

By yzi

Champion (438)

yzi's picture

13-11-2014, 10:48

Doesn't 60Hz interrupt rate mean adding over 16 ms of timing jitter? Latency is not so bad. Latency can be compensated for, and church organ players can live with huge latencies. Bug jitter is very bad because it ruins the musical content and it cannot be compensated.

By Grauw

Enlighted (7791)

Grauw's picture

13-11-2014, 12:54

hit9918 wrote:

Those "ld (0),a" and "cp 0" are about selfmodifying code. Faster than push / pop hl, ld hl,() ...

I’ll think about it. I’ve tried to avoided self modifying code in this project so far because I want to keep the code readable and easy to change. At the moment MIDI reception is fast enough (27 cycles slower than your code) and it’s better to work on a new feature than to spend a lot of time obfuscating my code Smile.

hit9918 wrote:

About line interrupts, grauw you mixed the pal vs ntsc line numbers. It is pal who has the 312 lines.

Ah, I had a feeling I might have. -_-;; Oopsie. In either case, NTSC can emulate 50Hz with line interrupts, but PAL can’t emulate 60Hz because it’d have to generate a line interrupt at line 260.

Not considering overscan, the highest practical rates are 300Hz for NTSC (every 52 lines) and 150Hz for PAL (every 104 lines). So when using line ints to generate a display frequency independent timer, 50Hz, 75Hz and 150Hz are the most practical control frequencies.

Btw the V9938 Application Manual says on page 18 (in Eugeny’s version) that PAL has 313 lines and NTSC 262. Those are both half of 625 and 525, but with different rounding... I wonder if the even and odd fields do not have the same number of lines?

hit9918 wrote:

A question is whether one can get more with opened borders.
One day I tried an int below vblank line and got inconistent things with bluemsx vs openmsx.
Someone with real machine would have to check out.

Yeah, I think it would work but it needs to be tried out.

yzi wrote:

Doesn't 60Hz interrupt rate mean adding over 16 ms of timing jitter? Latency is not so bad. Latency can be compensated for, and church organ players can live with huge latencies. Bug jitter is very bad because it ruins the musical content and it cannot be compensated.

I like to call it a "swing" feature Smile.

But I get what you mean. For applications where the clock is totally in sync like trackers or other kinds of prerecorded playback it is fine, but when sequenced from an external independently clocked device, the latency is going to vary.

From that perspective, if it’s going to jitter anyway, might perhaps as well run the control updates back-to-back in stead of on a fixed frequency to at least minimise the maximum... It will add jitter to ADSRs and LFOs though, don’t know how noticable that would be.

I’ve not noticed any real disturbing latency or jitter problems myself so far, but I’m not particularly sensitive to it, and especially when sequencing from a DAW I can see it could have some precision issues.

Question is, what is an acceptable level of jitter? In the end the limiting factor is the MSX CPU speed vs. the patch complexity, and probably the best way for me to deal with it is to just provide higher control frequency options as well as an unlimited one.

Page 4/8
1 | 2 | 3 | | 5 | 6 | 7 | 8
My MSX profile