Benchmark network speed

Page 2/3
1 | | 3

By sdsnatcher73

Paragon (1144)

sdsnatcher73's picture

23-09-2020, 06:17

Yes if you want to test TCP/IP pure throughput you should use a test client that discards the packets after receiving.

By S0urceror

Master (137)

S0urceror's picture

23-09-2020, 16:15

ducasp wrote:

TFTP isn't great for that as it is a protocol.... HGET to a local server is way better, even better if it doesn't save to disk

It would be great to have a diskless benchmark. Wanted to make a client for that as well. Also one that could switch between TCP and UDP to see the difference.

@duscasp, if you have a cool test client that also does the avg KB/sec display at the end we have a simpler repeatable test which I can use to upgrade the performance of both INL and my driver.

By ducasp

Champion (366)

ducasp's picture

23-09-2020, 16:51

Ok, so first, like you've asked, those are the results I have in my hand:

My ESP8266 firmware on an emulated SM-X Uart interface on a MSX2 @ 50Hz (BlueMSX) using ESE SCSI 1MB:

18,24s - 7KB/s

My ESP8266 firmware on an emulated SM-X Uart interface on a Turbo-R (BlueMSX) using ESE SCSI 1MB:

9s - 14,2KB/s

My ESP8266 firmware on a SM-X @ 3.58MHz:

15,2s - 8,42KB/s (transfer is much faster, Nextor takes a few seconds to create the file)

My ESP8266 firmware on a SM-X @ 8MHz:

7,2s - 17,8KB/s

This all on the interfaces, now with RAM Disk it is way slower

My ESP8266 firmware on a SM-X @ 3.58MHz:

30s - 4,27KB/s

My ESP8266 firmware on a SM-X @ 8MHz:

15,2s - 8,42KB/s

Now, BlueMSX emulates Obsonet, but, unfortunately the UDP emulation is a little bit broken as you receive the same UDP package multiple times, so this makes it useless to compare with a real Obsonet as the retries make it really slow.

My suggestion, please use my modified HGET that has a benchmark mode and Brunilda.ROM that is a 1248KB file, so a longer file will avoid buffering speed-ups (and I know, it can be tedious on a Obsonet):

hget http://192.168.1.1:8000/brunilda.rom /b

Of course, you need to change the address and port accordingly, an easy way to set-up a web server in linux box:

python -m http.server 8000 bind 192.168.1.1

(you need to change the address to the one that is assigned to your Linux box)

Another way, in Windows, is to use miniweb:

https://sourceforge.net/projects/miniweb/

You can get the HGET with /b option here:

https://drive.google.com/file/d/1u3PnfJ4AVHd5mQ-uBAcR4nvHaU2...

In that case, you will get a much closer to reality figure of raw Network performance:

My ESP8266 firmware on a SM-X @ 3.58MHz:

62KB/s

My ESP8266 firmware on a SM-X @ 8MHz:

78KB/s

My ESP8266 firmware on an emulated SM-X Uart interface on a MSX2 @ 50Hz (BlueMSX):

54KB/s

My ESP8266 firmware on an emulated SM-X Uart interface on a Turbo-R (BlueMSX):

54KB/s

Obsonet on MSX2 @ 50 Hz (BlueMSX):

5KB/s

Obsonet on Turbo-R (BlueMSX):

12KB/s

(I know, it could be really tedious to wait Brunilda transfer @5KB/s, over four minutes, and it might look like it is frozen, but it is not, just not spending any CPU time requesting to write to screen, so it is fully devoted to just transfer the data)

By ducasp

Champion (366)

ducasp's picture

23-09-2020, 18:22

S0urceror wrote:

For the development of the MSXUSB ethernet driver I would like to have an idea of the relative network performance.

For this I created a standard test that is hopefully reproducible with other hardware and other MSX-es:

  • MSX running Z80 on 3,58 Mhz
  • Ducasp's optimised TFTP client
  • TFTP server on my LAN (connected via Ethernet cable)
  • 128KB test file to download (used a typical Nextor firmware image but any 128Kb file is good)
  • TFTP server.lan GET image.rom

I get a speed of approximately 3.146 bytes/sec.

This is okay but I believe this can be much faster. So what are the numbers you're getting on GR8NET, BadCaT, ObsoNET, etc.?

P.S. This benchmark also depends on the speed it can save the file, I use a Carnivore2 + CF.
P.S. MSXUSB runs via InterNestor Lite implementing TCP/IP in software.

First things first:

You can download TFTP from Konamiman website as he has incorporated all my changes. HGET also incorporates all improvments, but the benchmark mode not, so to test only the network performance, I recommend getting HGET from my google drive link :)

Ok, now let's talk about your results:

As you can see, those are not bad results if you are using Internestor Lite... As you can see, in HGET, that is not ACK driven, Obsonet gets 5KB/s, and it is not even writting to the disk. My guess is that in the TFTP test it would get about the same result you got with your MSX USB... I don't think you are going to get much better results than that, and the reason is that all the network protocol is being driven by Internestor in the z80, sharing time with other tasks.

https://github.com/Konamiman/MSX/blob/master/SRC/INL/INL-RES...

As you can see, that is a lot of stuff that is a little bit heavy for the z80... Also, it is driven on the VDP interrupt (50 or 60Hz), so updates hapens 50 or 60 times a second. If we are talking about a 512 bytes package, that itself would limit the performance to at most 30KB/s receiving, but then you need to think that the package must arrive, it needs to be processed, and this is done every VDP tick... Then application need to check that there is data and read... If you think about writing it to disk, interrupts might be disabled for a while, and during that period the stack will not be running...

All in all, it is quite an achievment what Nestor has did to get such stack running on the MSX, resident... But z80 speed, all that need to be calculated, being driven by 60/50Hz interrupts that might take longer to occur during other system operations, and still have other stuff sharing the CPU time takes its toll...

As far as I understand, the only way to improve Obsonet (and probably your USB Ethernet) performance would be to re-work Internestor and optimize it even further, but also you need to be careful so you do not kill all the remaining system functions on the interrupts you are processing packets...

By S0urceror

Master (137)

S0urceror's picture

23-09-2020, 18:22

With your benchmark version of HGET I also did the Brunilda test.

MSXUSB via INL on a NMS8250 MSX2 @ 3.58MHz @ 50Hz:
6945 bytes/sec => 7KB/s

MSXUSB via INL on a NMS8250 MSX2 @ 3.58MHz @ 60Hz:
6797 bytes/sec => 7KB/s

MSXUSB via INL on a OpenMSX NMS8250 MSX2 @ 50Hz:
3651 bytes/sec => ~4KB/s

The last test is going via my USB testboard which is slow in OpenMSX.

The good news is that MSXUSB Ethernet is not so slow like Obsonet that also goes via INL. But way slower than hardware accelerated TCP. Shocked!

Thanks ducasp for your test and benchmark version of HGET, very much appreciated.

By Grauw

Ascended (9340)

Grauw's picture

23-09-2020, 18:26

Does it transfer packets serially via RS232, or via a memory buffer which can easily be LDIR-ed?

Because block operations is how you get high performance on a Z80…

By S0urceror

Master (137)

S0urceror's picture

23-09-2020, 18:35

Grauw wrote:

Does it transfer packets serially via RS232, or via a memory buffer which can easily be LDIR-ed?

On MSXUSB the packets come in via 8-bit parallel I/O. I use INIR to get consecutive bytes. Those are written into the buffer provided by InterNestor Lite via an UNAPI call.

But like ducasp writes, within InterNestor is where the magic happens. Packets are put 'in order' per connection and given to the client process like HGET/TELNET/TFTP etc. upon request. Maybe we can optimize INL more by using pointers to packet's vs moving them around with LDIR's, but I have to investigate that further.

By ducasp

Champion (366)

ducasp's picture

23-09-2020, 23:47

Just don't take BlueMSX numbers as an actual Obsonet result, it would be great if someone can test on an Obsonet, as the emulation not always reflect the real device performance (i.e.: SM-X Uart achieve higher speeds on SM-X than on emulated Turbo-R, and this surely is not true, Turbo-R should be at least as fast in R800 mode as probably the ESP speed to the UART is the bottleneck). Nevertheless, I agree your results are pretty good and USB / Ethernet w/ Internestor Lite is probably as good as Obsonet, if not better Cool

By S0urceror

Master (137)

S0urceror's picture

07-10-2020, 19:30

I have ported uIP to the MSX. This microIP stack is written in C for embedded devices.

Instead of HGET calling INL (UNAPI+TCPIP) calling USBETHER (UNAPI+ETHERNET) calling MSXUSB (UNAPI+MSXUSB).
It now goes from my new WGET command to MSXUSB (UNAPI+MSXUSB).

I now get a throughput of 8352 bytes/second. An improvement of 20%.

I expect more improvements to be done. I'll keep you posted on the developments. Once things are stable I will share uIP to all of you. Another IP stack for our beloved MSX. Cool

By S0urceror

Master (137)

S0urceror's picture

09-10-2020, 23:04

uIP on the MSX rocks.

With the TCP/IP checksums done in assembly instead of C I now get 11.113 bytes per second. A 60% improvement vs my initial measurements via Internestor Lite (INL).

Measurement is done by downloading BRUNILDA.ROM (1.277.952 bytes) in 115 seconds on my 3.58 MHz NMS8250 MSX2. Pretty impressive for a software based TCP/IP stack running on our trusted Z80.

uIP is a C-based IP stack that I compiled with the latest SDCC. uIP is statically linked into the client app. This makes the application around 22k bigger but it does everything in the main loop. During events callbacks are called to do something with received data or to handle errors. No dependency on reserved segments in memory mapper, background processing and interrupts. No INL, no UNAPI. Simple and straightforward.

I guess I can tweak it a bit more here and there but I'm satisfied with the results so far.

P.S. I'll put up the sources later this weekend. Then I'll start playing with the other clients like HUB and TELNET.

Page 2/3
1 | | 3