While working on the WiFi interface between a data collection system operating only through RS232 and a controlling system on a PC or Android phone that uses TCP/IP socket communications I have realized that there might exist a problem with data transfers to the RS232 based system.
So I want to ask here if the UART-bridge examples are able to handle data coming in on the TCP/IP link at a faster rate than it is possible to send it out the UART at 38400 baud?
The code examples I have found work in similar ways in loop() by checking if there are data on the incoming TCP connection and if so reads that and writes it to the serial port. Next the serial port is checked for incoming data which is read if existing and pushed out the TCP socket connection. Then loop() exits and is restarted by the underlying system.
I see no handling of buffers of data here, so my question is if this is done "under the hood" by the ESP8266 library such that the incoming data are "throttled" to a speed that can be handled by the outgoing UART? Or if data arrives at a faster pace it will be overrun and communication breaks?
As an example, imagine that the controller software needs to send a chunk of 2 Mbytes of binary data to the data collection system. This will need a transfer time of 8 min 40s over the serial connection. But the WiFi network from the controller software to the ESP8266 unit is orders of magnitude faster so it would probably need only a few seconds to dump the binaries onto the ESP. But the ESP does not have such a big buffer RAM, right?
Will this transfer be controlled by some method in the TCP/IP subsystem such that incoming data are only allowed at a rate compatible with the serial port baud rate?
If not, can someone suggest how it can be handled? Maybe some flow control inside the loop() function?
I was thinking along these lines too, such that the client socket would not ack to the sender until the data has been actually read off the inbound buffer. I.e. I would not read data from the TCP client until I could actually stuff it into the transmit buffer.
So then I would have to add some checks in the loop() function such that it will not read incoming data until the previous chunk has been actually sent so the UART transmit buffer (the Serial device) has room for more data.
Is there some way to check if Serial has space in its transmit buffer? And can one set the size of the transmit buffer of Serial?
check is availableForWrite(). Serial.write() implementation waits/blocks if the buffer is full.
uart.h has this line #define UART_TX_FIFO_SIZE 0x80 and it is used only to calculate the available size for availableForWrite(), so the real size must be set somewhere in SDK or it is a HW buffer
Juraj:
check is availableForWrite(). Serial.write() implementation waits/blocks if the buffer is full.
OK thanks, if Serial.write() blocks I guess that the loop() will stop and only proceed to the next call when the data read from TCP are actually completely put into the serial buffer. This would in effect slow down the transfers like I need, provided that there is no overrun on the incoming socket meanwhile. If a new packet arrives on TCP while loop() is blocked this will not be read and thus (hopefully) there will be no ack sent back and the sender will also wait.
I have planned to create a test application on my PC to test the data transfers for various sizes of payload, but it is a bit away yet...
Just to clarify, here is the code in loop() that checks incoming data destined for the Serial UART:
// Check for TCP client data ------
for (i = 0; i < MAX_SRV_CLIENTS; i++)
{
if (tcpServerClients[i] && tcpServerClients[i].connected())
{
//get data from the telnet client and push it to the UART
while ((bytesAvail = tcpServerClients[i].available()) > 0)
{
bytesIn = tcpServerClients[i].readBytes(buf, min(sizeof(buf), bytesAvail));
if (bytesIn > 0)
{
Serial.write(buf, bytesIn);
delay(0);
}
}
}
}
How could I add a check for the buffer state here (or is it actually needed)?
Also, since there is a while loop built into the data retrieval, will this cause the loop() to hang without ever processing the other stuff it is supposed to do as well, if the incoming data over TCP keeps filling the buffer faster than the UART can empty it?
As I read the response by @Juraj Serial.write(buf, bytesin) will only return when the bytes have actually been stored in the transmit buffer, so if meanwhile the tcpClients object has received more data the while loop will continue?
This should then lock up the system right here until there are no more data arriving on the TCP socket?
Maybe the while loop should be removed and instead get the next chunk of data in the next loop() execution?
There are other important tasks for loop() to deal with too...
UPDATE:
Now finally tested with an evaluation application I have written, that can stream both serial and tcp data towards the ESP8266 device.
It turns out that the buffering or throttle (whatever happens) is handled properly by the library. No lost data detected even using pretty large data blocks.
So I close this thread. (Don't know how to really close it though...)