Sending More than 90 Bytes at a Time WIFI Shield

Projects Networking, Protocols, and Devices
1

Hello Arduiono!

I have been working on my senior design project for a few weeks recently, my team and I are working
on implementing a wireless oscilloscope using the Due, and it has been a ton of fun so far!

Something we have been trying to figure out though, is how to increase the amount of data that can be sent via UDP in a single transmission. Right now we are calling udb.write() with a size of 90, and we are realizing connection speeds with average throughput of 400K bits/s on a private wireless network.

We want to see speeds of at least 10M bit/s on average, so we have our work cut out for us for now.

We have found this

ard_SPI.c
.....
#define _BUFFERSIZE 100
.....

We have experimented with making this larger, recompiling and loading the firmware, but it doesn't
seem to make a difference.

Has anyone figured out how to get past the 90 byte/transmission limited, or have found other ways to drastically
increase the connection speed?

Thanks :slight_smile:

Oh, if we manage to work out how to increase the wireless speed, we wouldn't mind sharing the information with all !

2

Yep. A 90 byte payload limit is pitiful.
Ethernet header = 24 bytes
ipV4 header = 20 bytes
UDP header = 8 bytes
= 52 bytes.

So not only is the payload tiny but sending more than a single packet is ludicrously inefficient, to the point I would call it unusable.

I have had a scan through the source and there appears to be no obvious reason for the limit. At the lowest level of the stack, a 1024 byte payload is supported as far as I can tell. I will be very surprised if the 90 byte limit does not turn out to be a bug.

_BUFFERSIZE appears to relate to the SPI receive buffer.

What I have found browsing the code bottom up, is this.

//ard_spi.c 
send_data_udp_cmd_cb(int numParam, char* buf, void* ctx) {
//...
        uint8_t* p = mergeBuf(sock, NULL, &len);
        err = sendUdpData(getTTCP(sock, TTCP_MODE_TRANSMIT), p, len);
//...
}

// ard_utils.c  
uint8_t* mergeBuf(uint8_t sock, uint8_t** buf, uint16_t* _len) {
uint16_t len = calcMergeLen(sock);
//...
} 

uint16_t calcMergeLen(uint8_t sock){
//...
}

However, the Atmel and H&D code appears to be well documented and well maintained.

Browsing top down, from the not so well maintained WiFiUDP.h file I ran into a suspicious line of source.
My words follow the semicolon

//file WiFi/utility/spi_drv.cpp
void SpiDrv::sendCmd(uint8_t cmd, uint8_t numParam)
{
//..

    // Send Spi totLen
    //spiTransfer(totLen);     <--- this line is commented out, should it be?

//..
}

If you do find a way around the limit, please do share as I am sure a lot of people including me, would be eternally grateful.

3

MattS-UK:
I have had a scan through the source and there appears to be no obvious reason for the limit. At the lowest level of the stack, a 1024 byte payload is supported as far as I can tell. I will be very surprised if the 90 byte limit does not turn out to be a bug.

It may well be a bug, but remember that a lot of the processors only have 2 kB of RAM, so don't get too excited. A 65536 byte payload won't be easily achievable.

4

I use a 64 byte buffer for SD to internet transfers with the ethernet shield. I know this isn't an ethernet shield, but the speed limitation is the same. That is the SPI bus speed. With a Mega, the max SPI clock speed is 8MHz, and the w5100 requires more than a one byte SPI transfer for a one byte tcp/udp send. The datasheet shows 4 bytes transferred over the SPI bus for one data byte (one command byte, two address bytes, and one data byte). That means at max, you will get 2MHz data transfer. I imagine the HDG104 wifi IC is not that much different.

Your best bet to increase sending speed is increase the SPI bus speed. The HDG104 seems to be able to handle a higher SPI clock speed than the w5100, as does the Due.

5

Glad to see we are starting to do some research into this!

When I get into the lab today, my partner and I will trace out and diagram the entire program flow sequence in Lucid
Chart and will post a link to it here.

Something we have been trying to figure out is what is the best way to retrieve the error codes that are set inside of the driver?. When we try and retrieve the error from the UDP class and print it over serial it's just printing blank data, might want to try and print it out as hex maybe.

I do know that the SPI bus can on the DUE can run at the system clock which would be at 84MHZ, and the radio chip itself can
handle 100 MHZ in slave mode from what I remember. I will look into that.

6

As a network engineer who occasionally programs, I am more concerned to see applications and devices sharing the network nicely. In my day job I have had to have soo many chats with developers who forgot network resources are not unlimited :wink:

A 65536 payload will be broken down (fragmented) by the network. The typicalIP network MTU size is 1500 bytes. Although that figure is not set in stone, it is a de-facto standard for most intents and purposes.

Roughly speaking, a 65536 payload would be broken into ~46 network packets, each packet containing ~1425 bytes of data and 52 bytes of header overheads. Which produces an efficiency number of ~27:1
A 1024 byte payload limit would produce an efficiency number ~20:1, which is not too bad.
The 90 byte payload limit results in an efficiency number ~2:1, a whole order of magnitude lower.

Start putting routers in the path and relatively large amounts of resources are consumed and unnecessary latency generated, by a relatively tiny volume of data.

7

On this page there are a number of complaints about the 90 byte limit:

No responses as far as I can see.

8

I had a chance to look at the problem again yesterday

Tracing a successful send of a 90 byte packet and a failed 91 byte packet.
I found a problem occurs within the SPI diver class, during a call to,
WiFiUDP::Write(uint8_t*, size_t);

Reference code below. You can see the debugState object I inserted
For a successful 90 byte write,
debugState.checkStart = 1
debugState.checkCmd = 1

For a failed 91 byte write
debugState.checkStart = 1
debugState.checkCmd = 0

IF_CHECK_START_CMD
Is a nested macro but expanding it proved to be beyond me and I don't know a great deal about SPI anyhow.

//spi_drv.cpp
int SpiDrv::waitResponseData8(uint8_t cmd, uint8_t* param, uint8_t* param_len)
{
	char _data = 0;
    int ii = 0;
	
	//dbg
    debugState.checkStart = 1;
		
    IF_CHECK_START_CMD(_data)
    {
		//dbg
		debugState.checkCmd = 1;
		
		CHECK_DATA(cmd | REPLY_FLAG, _data){};
				
        uint8_t numParam = readChar();
        if (numParam != 0)
        {        
            readParamLen8(param_len);
            for (ii=0; ii<(*param_len); ++ii)
            {
                // Get Params data
                param[ii] = spiTransfer(DUMMY_DATA);
            } 
        }         

        readAndCheckChar(END_CMD, &_data);
    }     

    return 1;
}
9

It appears those macros get down to this function in spi_drv.cpp

int SpiDrv::waitSpiChar(unsigned char waitChar)
{
    int timeout = TIMEOUT_CHAR;
    unsigned char _readChar = 0;
    do{
        _readChar = readChar(); //get data byte
        if (_readChar == ERR_CMD)
        {
        	WARN("Err cmd received\n");
        	return -1;
        }
    }while((timeout-- > 0) && (_readChar != waitChar));
    return  (_readChar == waitChar);
}

There is a timeout. It is defined in wifi_spi.h

#define TIMEOUT_CHAR    1000

Maybe the timeout isn't long enough if the transfer is longer than 90 bytes?
Or it received an ERR_CMD?

edit: I did not check, but if the readChar() function is like the Serial.read() function, it is non-blocking. It returns immediately but returns int -1 if no character available. If that is the case, that function could really roll through that timeout value.

10

Thanks Tim.

The latest instalment

Maybe the timeout isn't long enough if the transfer is longer than 90 bytes?
Or it received an ERR_CMD?

Not the timeout or the ERR_CMD.

When you send a 90 byte packet, during the call to WiFiUDP::Write,
SpiDrv::waitSPIChar(waitChar) waits for and receives END_CMD (0x0E).

When you send a 91 byte packet, during the call to WiFiUDP.Write,
SpiDrv::waitSPIChar(waitChar) waits for END_CMD but receives START_CMD (0xEE),
causing the do...while loop to timeout.

I fudged the SpiDrv::waitSPIChar(...) function to exit the loop when the START_CMD was received and to return true.
The fudges made no real difference.

Where to look next I wonder?

11

That function is pretty much the last function before the actual read/write over the SPI bus, so I would think the error is in the wifi shield firmware, not the library, but that is just a guess.

// This function...
int SpiDrv::waitSpiChar(unsigned char waitChar)
{
    int timeout = TIMEOUT_CHAR;
    unsigned char _readChar = 0;
    do{
        _readChar = readChar(); //get data byte
        if (_readChar == ERR_CMD)
        {
        	WARN("Err cmd received\n");
        	return -1;
        }
    }while((timeout-- > 0) && (_readChar != waitChar));
    return  (_readChar == waitChar);
}

// calls this function...
char SpiDrv::readChar()
{
	uint8_t readChar = 0;
	getParam(&readChar);
	return readChar;
}

// which calls this function...
void SpiDrv::getParam(uint8_t* param)
{
    // Get Params data
    *param = spiTransfer(DUMMY_DATA);
    DELAY_TRANSFER();
}

// which calls this function.
char SpiDrv::spiTransfer(volatile char data)
{
    SPDR = data;                    // Start the transmission
    while (!(SPSR & (1<<SPIF)))     // Wait the end of the transmission
    {
    };
    char result = SPDR;
    DELAY_TRANSFER();

    return result;                    // return the received byte
}
12

So digging through the code, I just confirmed in my mind what you guys pointed out already. This timeout is based on the number of iterations through a loop, not an actual time dependency. Something that is worth noting is this though, because it could come into play later when we start taking a look into the MCU for the Wifi Chip.

Every time we call getParam() we call spiTransfer(), spiTransfer() then transmits a dummy byte, listen, then SPINS FOR 10 CYCLES. When we end up back inside getParam(), we then SPIN FOR 10 CYCLES. So we are spinning for 20 CYCLES every time we call getParam(). In the current code, we call getParam() 1000 times, meaning we are waiting for a total of 20,000 CYCLES.

Might not be related, but it was something I observed going through the code. I will now continue looking at the driver code for the MCU on the WIFI chip.

13

If we are indeed having problems with actually receiving the message START_CMD instead of END_CMD, I might have
found something that could get us started:

Inside of wifiHD/src/ard_utils.h

I found these macros.

#define CREATE_HEADER_REPLY(REPLY, RECV, NUM_PARAMS)\
  REPLY[0] = RECV[0];                           \
  REPLY[1] = RECV[1] | REPLY_FLAG;            \
  REPLY[2] = NUM_PARAMS;

#define END_HEADER_REPLY(REPLY, TOT_LEN, COUNT)\
    REPLY[TOT_LEN] = END_CMD;           \
    REPLY[TOT_LEN+1] = 0;               \
    COUNT=TOT_LEN+1;

These are called to place data inside of

static char reply[REPLY_MAX_LEN];

which is declared inside of wifiHD/src/ard_spi.c

I am wondering if we entered some kind of state where CREATE_HEADER_REPLY was called and END_HEADER_REPLY, was never called, or CREATE_HEADER_REPLY was called after END_HEADER_REPLY was called. Which could explain why we received the START_CMD, but never END_CMD.

So, I think we need to figure out why this code here, has the wrong data inside of "reply" when it is called

int sendReply(int cmdIdx, char* recv, char* reply, void* resultCmd)
{
	uint16_t _count = 0;
    int _result = SPI_OK;

    cmd_spi_list[cmdIdx].reply_cb(recv, reply, resultCmd, &_count);
    state = SPI_CMD_REPLING;

    AVAIL_FOR_SPI();
    _result = write_stream(ARD_SPI, &reply[0], _count);
#ifdef _SPI_STATS_
    if ( _result != SPI_OK)
    {
    	statSpi.lastCmd = cmd_spi_list[cmdIdx].cmd_id;
    }
#endif
    BUSY_FOR_SPI();

    IF_SPI_DUMP(printk("==>"));
    DUMP_SPI(recv, count);
    IF_SPI_DUMP(printk("<=="));
	DUMP_SPI(reply, _count);
    replyCount = _count;
    return _result;
}