Fast way to get data TO my Due? (RX is slow, TX is fast)

Hi there,

I've have Due board into which I need to get data from my PC fast (for a display).

First, I tried connecting with Ethernet using the Shield (w5100).
When using a TCP connection I get a 55kbits/s throughput (not storing the data anywhere) using the "Ethernet" library. That's very slow! I suspect the single-byte "EthernetClient.read()"-function being the slow part here.

Then, I tried sending data as UDP. That's a lot faster - here I get 450kbits/s. And here we have a multibyte read-function, "Udp.read()".

Finally, I turned to the Native USB using the "SerialUSB" library. Again, here we have a multi-byte read-operation "SerialUSB.readBytes()", and here I can get the best throughput of 1.0mbits/s.
However, 1 mbs is still not impressive, so I tried sending data to my Windows host instead (SerialUSB.print(char[])). That was impressive: 26.5 mbits/s.

So, my question, is there a way to get data just as fast (or just faster) to my Arduino?

Searching for answer, I read somewhere that others with the Ethernet Shield (w5100) could only do some 0.3mbs (I at least got 0.45mbs) [DUE ethernet performance on W5200 with SPI and SPI+DMA - Arduino Due - Arduino Forum].

BR,
Egholm

I recently did some USB receive benchmarking. I saw a similar, approx 110 kbytes/sec speed. Here's the results.

http://www.pjrc.com/teensy/benchmark_usb_serial_receive.html

Due's USB should be capable of much faster.

I also started some work on optimizing the Ethernet library for the W5200 chip, which is a newer version of the W5100 used on most shields today. I'm sad to say the current Ethernet library is extremely slow. Just speeding up the SPI does little good. I'm working on some optimizations to use local buffering, so the library will be able to really take advantage of the W5200's block read speed and satisfy most reads from a buffer without the substantial overhead of slowly querying lots of the chip's registers every time.

Ah, a very nice benchmark page. So, now I have a source for both my USB and and UDP results. That is somewhat comforting :-/
So, should I go ahead and order a Teensy 3.0, or is this new Ethernet library (w5200) in such a good shape that I should go that way?

Thanks...

The work with the W5200 is at a very early stage, definitely not usable yet. I hope to publish a usable version in a few weeks.

The W5200 has a low-level protocol that's theoretically capable of dramatically faster speed than the W5100, but the inefficient way the Ethernet library is currently structured makes the W5200 run even slower than the W5100. Changing the middle layers of that library is not a simple or easy task.....

Teensy3 may or may not work for your display needs. It does have a very optimized USB stack. For full disclosure, I'm the author.... so my opinion may be biased.

On the Ethernet low-level speeds, the W5100 always has 75% overhead. Every 1 byte access has 3 overhead bytes. So in a best case scenario, the fastest it can achieve is 25% of the SPI speed.

Of course, with the current Ethernet library, there is massive overhead accessing the W5100 registers (most are 16 bits) many times for every 1 byte of data read.

The W5200 has 4 bytes of overhead, compared to only 3 bytes on the W5100. But each transfer can be any size (up to 32 kbytes). So for single byte transfer, it has even more overhead than the W5100. For 2 bytes, it becomes "only" 67% overhead. For larger sizes, the overhead becomes pretty small.

Sadly, the Ethernet library does nearly everything as single byte transfers, even when reading 16 bit registers!

The exception is UDP, where block read is used to good effect. The one alternative that might be very fast with Arduino Due is a W5200 using UDP. Currently the only W5200 support is a simple modification to the existing library published by the Wiznet folks (who make the chip). It does use the efficient block transfer, but does nothing to optimize higher levels in the library. Since UDP is already pretty efficient, that might be reasonably fast, especially if you do some hacks to increase the SPI clock speed.

Biased or not - the benchmarks kinda talks for themselves.
For my solution I will need a fast SPI library for communicating with the display controller, as well.
I'll investigate.

[quote author=Paul Stoffregen link=topic=189702.msg1403399#msg1403399 date=1380026171]
On the Ethernet low-level speeds, the W5100 always has 75% overhead. Every 1 byte access has 3 overhead bytes. So in a best case scenario, the fastest it can achieve is 25% of the SPI speed.[/quote]
Yikes.

So what you say is that UDP with the w5200 is easy to integrate? At least without further optimizations?

Thanks

ArduinoEgholm:
So what you say is that UDP with the w5200 is easy to integrate? At least without further optimizations?

Well, no, not exactly. I'm saying it might be faster, when used with Due and the existing (not very well optimized) Ethernet files for W5200. Maybe?

I haven't personally tested this. But if you get a W5200 (eg, a Wiz820io module) and connect it to an Arduino Due and replace w5100.cpp and w5100.h with the W5200 versions (as published by Wiznet), I'd be curious to hear how fast it actually is?

When I publish a modified Ethernet library with speed improvements, I'll probably publish a benchmarks page. But that's still very much a work-in-progress that's unlike to be ready within the next few weeks, and very likely could be delayed for months, maybe even into next year. It's a lot of work, and of course it compete for time with all the other stuff I'd doing.....

:slight_smile: Ok, I hear you - it might work.
Thanks a lot for the input!

[quote author=Paul Stoffregen link=topic=189702.msg1403529#msg1403529 date=1380031541]
Well, no, not exactly. I'm saying it might be faster, when used with Due and the existing (not very well optimized) Ethernet files for W5200. Maybe?

I haven't personally tested this. But if you get a W5200 (eg, a Wiz820io module) and connect it to an Arduino Due and replace w5100.cpp and w5100.h with the W5200 versions (as published by Wiznet), I'd be curious to hear how fast it actually is?[/quote]

Hi Paul,
I got myself a W5200 Shield (http://www.shopwiznet.com/w5200-ethernet-shield) and tested with the W5200 versions of the w5100.c|h files.

The TCP performance dropped, as you prediceted due to the overhead. I'm not sure exactly how much, but somewhere between 50% and 75% (give and take - I urged to UDP, and am not at my desk now).
The UDP, however, went from 450kbps (w5100) to 750kpbs (w5200). Still lower than my USB-native, though.

I would love to get hold of the code used obtaining these numbers - 15.6Mbps UDP, 12.9Mbps TCP:
http://wizwiki.net/w5200-performance-test-with-arduino-due-maple-and-teensy3-0/

BR,
Egholm

And changing the SPI clock divider to 2 increased the speed to 1.8Mbps.
However, still a little far from the 15.6Mbps.

I was reading the w5200 iEthernet datasheet, and it says that the maximum SLK Clock Frequency is 80MHz, with a minimum High/Low point of 6 nanoseconds, and 80MHz point is 12 ns (there probably is a small delay when it switches between high and low). I don't know if the Arduino Due SPI can clock that fast, or what's the issue with raising the clock that fast?

If you go to his github, where they made that performance test, you'll notice they have some code tests written. It seems the guy tried several different tests;
-Increasing the 28MHz clock to 42MHz, caused errors. (Doesnt say what exactly happened for it to error out.)
-Using just SPI vs. SPI & DMA
-Using SPI Extended (SAM3X) vs. Arduino SPI
-Using SPI Extended (?) + DMA << Best?

It looks like he wrote a w5200 fork that is based off of the w5100 source code;