Stack overflow while using H.KEYI hook

Page 2/4
1 | | 3 | 4

By tvalenca

Paladin (747)

tvalenca's picture

02-09-2020, 16:17

pgimeno wrote:
tvalenca wrote:

You're telling me I should use JP instead CALL on this hook?

Yes. Or on any hook, really. If you really want to place a CALL instead of a JP, be prepared to add a RET afterwards, just in case someone else was already using the hook. It's just a waste because it's a tail call, so it can be safely replaced with a JP and save a few cycles and stack space.

It's not likely to be the issue, though, because it's not likely that the hook was in use before you overwrite it, therefore the hook is probably filled with #c9's (RET's), and therefore there's already a RET immediately after your call anyway. But better not leave anything to chance.

Well, you do make sense. Since I am using H.KEYI I should RETurn somewhere, BUT, if (like Grauw said earlier) something else was hooked on it I couldn't simply JumP, that's why I used CALL instead. (yes, I know that I must check if something else is using this hook before overwritting #FD9A)

By tvalenca

Paladin (747)

tvalenca's picture

02-09-2020, 16:36

ducasp wrote:

Thiago, this code might help you, first to register into HKEY_I hook:

Wow, do you know me? Sorry, I can't remember you. :-( At least not by this nickname.

ducasp wrote:
	ramad1		equ	0xF342

	di
	ld		(intr_handler+1), hl	; Replace in intr_handler the address
	;	save hookint 
	ld		hl, #hookint
	ld		de, #backup_hookint
	ld		bc, #5
	ldir
		
	ld		a, #0xF7				; RST_30 (interslot call both dos and bios)
	ld		(hookint), a
	ld		a, (ramad1)
	ld		(hookint+1), a
	ld		hl, #intr_handler
	ld		(hookint+2), hl
	ld		a, #0xc9
	ld		(hookint+4), a
	ei

backup_hookint:
	.ds		5

This might help you, even though it is intended to use interslot call, you can replace it by call to a P3 address if your routine is there, instead of using it, then you won't need ramad1 (slot of ram in page 1, which is the program ram in case of DOS, you might want to change to ramad2 depending on your needs, i.e.: if it is something running in BASIC, it will be running on page 2). It is important to backup the original hook 5 bytes so you can call it after doing your interrupt, you need to chain in this way, otherwise, it won't be nice to any other piece of BIOS/ROM/DRIVER that is hooking there and will no longer get called on interrupts... By the way, this code assumes HL has the address of your HKEY_I routine...

is this some kind of hook installer that don't trashes already installed hooks? Neat!

ducasp wrote:

At the end of your program, you can try to get yourself out of the hook by doing this, but only if you are running a program, if you are doing resident stuff (i.e.: allocating a system mapper segment that gets called) this might not work as there is a good chance you are no longer the last to hook:

	di
	ld		hl, #backup_hookint
	ld		de, #hookint
	ld		bc, #5
	ldir
	ei
	ret
intr_handler:
	call	dummy_handler		; This will be replaced by user routine address
dummy_handler:
	jp		backup_hookint		; And continue processing interrupt hooks
	ret

And this is your code executing whenever an interrupt is called... As you can see, this is as generic as can be (this was my proposition, which I had help from DarkSchneider, for functions to register a SDCC function to HKEY_I interrupt). In your case, you might just erase the first LD after DI in the install routine, and below intr_handler put all your interrupt code, just make sure it ends with jp backup_hookint to execute the previous hook and has a ret below, so you will return after the backup hook returns...

This works quite well with SDCC and should work on your code as well. Wink

Well... this code is meant to be "resident stuff" (it will grow to animate tiles and palettes, it already has a very simple V9990 splitscreen code) so it should RETurn normally.

By Grauw

Ascended (10062)

Grauw's picture

02-09-2020, 18:23

tvalenca wrote:

is this some kind of hook installer that don't trashes already installed hooks? Neat!

Yes, it is the standard pattern of how a hook should be set up. Copy the original 5 hook bytes to another location in RAM, and then jump to it at the end of your own hook code. This way it forms a chain of hooks.

By ducasp

Champion (461)

ducasp's picture

02-09-2020, 20:25

tvalenca wrote:
ducasp wrote:

Thiago, this code might help you, first to register into HKEY_I hook:

Wow, do you know me? Sorry, I can't remember you. :-( At least not by this nickname.

Hi, yeah, Facebook MSX Group, I'm Oduvaldo, the guy behind MSX Telnet and SM-X WiFi Smile

tvalenca wrote:

Well... this code is meant to be "resident stuff" (it will grow to animate tiles and palettes, it already has a very simple V9990 splitscreen code) so it should RETurn normally.

By resident, I mean a code that is running besides another application, i.e.: a ROM DISK, an UNAPI ROM or driver... Those codes only execute to install it's hooks (both hooking to hooks or creating their own hooks, i.e.: UNAPI hook, EXTBIO, etc) and initialize everything, then that code will be running along with other applications, being either invoked by a hook, interrupt, etc... In that case, as you install your stuff and system continues with other applications, someone else might hook after you and if you simply remove your hook, you kill the hook chain after you, for resident applications, if you wan't to disable your hooking, instead of removing yourself from the hook, you just make your hook code to jump directly to your hook backup, so it doesn't break the hook chain...

If your code will not, ever, exit and return control, or if when it exits it is done and no part of it is kept running, it is not resident (i.e.: games, most dos applications, etc). In that case, just remove your hook entry before exiting, so the system won't crash once your program is unloaded or overwritten in the memory Wink

By tvalenca

Paladin (747)

tvalenca's picture

03-09-2020, 01:57

ducasp wrote:

(...)By resident, I mean a code that is running besides another application, i.e.: a ROM DISK, an UNAPI ROM or driver... Those codes only execute to install it's hooks (both hooking to hooks or creating their own hooks, i.e.: UNAPI hook, EXTBIO, etc) and initialize everything, then that code will be running along with other applications, being either invoked by a hook, interrupt, etc... In that case, as you install your stuff and system continues with other applications, someone else might hook after you and if you simply remove your hook, you kill the hook chain after you, for resident applications, if you wan't to disable your hooking, instead of removing yourself from the hook, you just make your hook code to jump directly to your hook backup, so it doesn't break the hook chain...

In my MS-DOS days, this type of program were called "TSR" (Terminate and Stay Resident). Yes, that's exactly what I am doing here.

ducasp wrote:

If your code will not, ever, exit and return control, or if when it exits it is done and no part of it is kept running, it is not resident (i.e.: games, most dos applications, etc). In that case, just remove your hook entry before exiting, so the system won't crash once your program is unloaded or overwritten in the memory Wink

It may happen in the future Smile

By pgimeno

Champion (300)

pgimeno's picture

03-09-2020, 06:41

tvalenca wrote:

Well, you do make sense. Since I am using H.KEYI I should RETurn somewhere, BUT, if (like Grauw said earlier) something else was hooked on it I couldn't simply JumP, that's why I used CALL instead. (yes, I know that I must check if something else is using this hook before overwritting #FD9A)

Sorry, I must not be explaining myself very clearly. Let me try again.

The system hooks are all 5 bytes long. The system initializes all hooks with #c9's (RET instructions). The interrupt routine calls H.KEYI via a normal call instruction, and expects control to return to the instruction following that call instruction. This is how the ROM code calls H.KEYI:

                CALL H.KEYI

Just a standard call to the H.KEYI address (FD9A). It expects the code in there to do whatever it needs to do and then perform a RET, so that the rest of the BIOS interrupt routine can resume execution. If nothing else installed a hook, the FD9A address already contains a RET, therefore the code can just continue.

Whenever someone hooks on H.KEYI, they can install a small routine using a maximum of 5 bytes starting at FD9A. This is designed to allow for an interslot call (RST 30h + slot + address) followed by a RET, which uses up exactly 5 bytes (or anything shorter than that, of course). This routine must be relocatable (there isn't much space to place a non-relocatable routine there anyway).

Now, let's assume that some program has placed an interrupt routine in H.KEYI before your program executes. For dramatic effect, let's say that the interrupt routine is an interslot call to slot 2 and address C700. This means that BEFORE your program executes, the contents of H.KEYI will be:

FD9A  F7        RST  30   ; Inter-slot call
FD9B  02        DB   02   ; Slot number
FD9C  00 C7     DW   C700 ; Address
FD9E  C9        RET       ; Return to caller

Now your program comes and installs another H.KEYI handler. But the way you've written your code, you're only replacing the first three bytes, which before you overwrite them, correspond to F7 02 00, and your program is not touching the C7 or the C9. See how the H.KEYI hook looks like after you install it with a CALL:

FD9A  CD 00 C0  CALL C000   ; Call to your code
FD9D  C7        RST  00     ; Reset?!?!
FD9E  C9        RET         ; Return (never reached)

So, when the ROM calls the H.KEYI hook, this part will call your routine and then... perform a RESET! (see why I said "for dramatic effect"? I've chosen the high part of the address to correspond to a RST 00 opcode).

This will not happen if you place a C3 (JP) instead of a CD (CALL) opcode:

FD9A  C3 00 C0  JP   C000   ; Jump to your code
FD9D  C7        RST  00     ; Reset (never reached)
FD9E  C9        RET         ; Return (never reached)

It will also not happen if you add a C9 byte after CD 00 C0, which is pretty much equivalent but wastes some bytes and cycles, but overwrites the C7:

FD9A  CD 00 C0  CALL C000  ; Call your code
FD9D  C9        RET        ; Return to BIOS
FD9E  C9        RET        ; Return (never reached)

The way to chain back to the previous handler is to copy the 5 bytes starting at FD9A to some other place. Imagine this place is C100. Then, at the end of your handler, instead of returning, you jump to C100.

This is the layout BEFORE installing your handler, assuming someone else hooked it as before:

ROM:
                CALL H.KEYI
                ...

H.KEYI:
FD9A  F7        RST  30   ; Inter-slot call
FD9B  02        DB   02   ; Slot number
FD9C  00 C7     DW   C700 ; Address
FD9E  C9        RET       ; Return to caller

Slot 2 address C700:
02:C700 ... (previous handler does its stuff here)
                RET

The BIOS calls H.KEYI. H.KEYI calls C700 in slot 2. C700 does its stuff and returns (to FD9E). Then H.KEYI returns (to the BIOS).

And this is the layout after you install your handler, assuming you have first copied 5 bytes from H.KEYI to C100:

ROM:
                CALL H.KEYI
                ...

H.KEYI:
FD9A  C3 00 C0  JP   C000   ; Jump to your code
FD9D  C7        RST  00     ; Reset (never reached)
FD9E  C9        RET         ; Return (never reached)

C000 ... do your stuff here
        JP    C100 ; Chain back to the previous handler

C100  F7        RST  30   ; Inter-slot call
C101  02        DB   02   ; Slot number
C102  00 C7     DW   C700 ; Address
C104  C9        RET       ; Return to caller

Slot 2 address C700:
02:C700 ... (previous handler does its stuff here)
                RET

The BIOS calls H.KEYI, which jumps to your code. Your code does its stuff and then jumps to C100, which executes the 5 byte routine that was present in the previous hook, which does an inter-slot call to 02:C700. This routine does its own stuff and returns to C104, which in turn returns to the BIOS; execution continues normally and everyone's happy.

If someone hooks after you, they will copy your JP to their own area and then replace the hook. They will execute their stuff, then your JP, which will call your code, which will do your stuff, then you execute the previous hook's code and so on until the last one in the chain returns to BIOS (because the hook initially contains only RETs).

By tvalenca

Paladin (747)

tvalenca's picture

03-09-2020, 14:36

still don't get the "dramatic effect" part. WHY it would RST_0 if I didn't put it there anyway? My approach to write only the first 3 bytes was because I am only testing right now (I only pulled this out a long time ago: https://youtu.be/AdRgkHJtLt8 - source code may be on this thread)

Anyway, great post! I got some fresh ideas from it.

By Grauw

Ascended (10062)

Grauw's picture

03-09-2020, 15:05

Because C7 is the opcode for RST 0. In the example you write a call (CD nn nn) to the hook instead of a jp (C3 nn nn), without writing a ret (C9) after the call. So as the call returns it will execute whatever was originally there, in pgimeno’s example case that’s an RST 0 (reset). Often the hook is filled entirely with rets, however it is not guaranteed, so this means there is a hidden bug that you would probably overlook.

You need to copy the old hook to another location in RAM, jump to your ISR instead of call, and at the end of your ISR code jump to the old hook to make sure it is executed (or the system may crash).

By tvalenca

Paladin (747)

tvalenca's picture

03-09-2020, 17:45

Grauw wrote:

Because C7 is the opcode for RST 0. In the example you write a call (CD nn nn) to the hook instead of a jp (C3 nn nn), without writing a ret (C9) after the call. So as the call returns it will execute whatever was originally there, in pgimeno’s example case that’s an RST 0 (reset). Often the hook is filled entirely with rets, however it is not guaranteed, so this means there is a hidden bug that you would probably overlook.

But I don't have a single C7 (either instruction, data or memory location) on my code. Maybe it was related to pgimeno's example. On my tests, there wasn't another hook instaled prior to my hook install routine, so, it should find a ret instead. but instead resuming, the code is still loosing itself and pushing anything on stack, corrupting RAM in the process.

Grauw wrote:

You need to copy the old hook to another location in RAM, jump to your ISR instead of call, and at the end of your ISR code jump to the old hook to make sure it is executed (or the system may crash).

That I understood.

By ducasp

Champion (461)

ducasp's picture

03-09-2020, 20:28

tvalenca wrote:
Grauw wrote:

Because C7 is the opcode for RST 0. In the example you write a call (CD nn nn) to the hook instead of a jp (C3 nn nn), without writing a ret (C9) after the call. So as the call returns it will execute whatever was originally there, in pgimeno’s example case that’s an RST 0 (reset). Often the hook is filled entirely with rets, however it is not guaranteed, so this means there is a hidden bug that you would probably overlook.

But I don't have a single C7 (either instruction, data or memory location) on my code. Maybe it was related to pgimeno's example. On my tests, there wasn't another hook instaled prior to my hook install routine, so, it should find a ret instead. but instead resuming, the code is still loosing itself and pushing anything on stack, corrupting RAM in the process.

Grauw wrote:

You need to copy the old hook to another location in RAM, jump to your ISR instead of call, and at the end of your ISR code jump to the old hook to make sure it is executed (or the system may crash).

That I understood.

You are focusing on the wrong stuff... It doesn't matter if you have a C7 after CD xx yy, what matters is if you DO NOT HAVE a C9 (ret) after your CD xx yy... Because if you don't, once your HKEY_I routine returns, it will execute whatever is in front of CD xx yy, and thus, there is a good chance that it will execute some strange code.

Because:

CALL MYHKEYROUTINE
??
??
??
??

So once it returns from MYHKEYROUTINE, it will execute ?? ?? ?? ?? that can lead to unpredictable results

So you need to have in the HOOK:

CALL MYHKEYROUTINE
RET
?? (doesn't matter what it is)

Pigmeneo just told you that if you change from CALL to JP, that would work fine even without the RET after JP XX YY because:

Interrupt -> Routine at 0x0038 is run, at some point that routine will execute:

CALL H_KEYI

Which will:

JP MYHKEYROUTINE

And MYHKEYROUTINE ends with:

RET

That, since a JP was used to go to it, will return to the routine at 0x0038, just after the CALL H_KEYI...

Page 2/4
1 | | 3 | 4