Unstable client.write with TCP connection

Projects Networking, Protocols, and Devices
1

I have a program that scans a 32x32 matrix measures the data and sends it over wifi to my pc. The problem is that the client.write is taking varying amounts of time to send the data. here are the send times:

image
image351×633 3.54 KB

I tried to send the data thru a 300M mini smart router that is not connect to the internet. I power the esp32 with usb so i tried to directly power it with 5v but its still unstable. i suspect the issue is in the arduino code. I wonder since i use TCP the problem could also be in the python code? but i dont know, iam new to this.

This is my arduino code for sending the data(the timer is over the send data function being used):

// Function to send all data in packets
void sendData(uint8_t* buffer, int length) {
  int bytesSent = 0;

  while (bytesSent < length) {
    int packetSize = min(maxPacketSize, length - bytesSent);
    memcpy(packetBuffer, &buffer[bytesSent], packetSize);

    if (client.connected()) {
      unsigned long startTime = millis();
      size_t sent = client.write(packetBuffer, packetSize);
      client.flush();
      unsigned long elapsedTime = millis() - startTime;
      Serial.println("Send time for packet: " + String(elapsedTime) + " ms");
      if (sent != packetSize) {
        Serial.println("Failed to send full packet. Retrying...");
        continue; // Retry sending the same data
      }
      Serial.println("Bytes sent: " + String(sent));
    } else {
      Serial.println("Client disconnected. Reconnecting...");
      client.stop();
      while (!client.connect(host, port)) {
        Serial.println("Reconnecting to server...");
        delay(2000);
      }
      client.setNoDelay(true); // Reapply TCP_NODELAY after reconnecting
    }

    bytesSent += packetSize;
  }
}

Any tips would be appreciated, Thank you!

2

What is the typical length of the total payload you are sending in "packets"?

I have a test web server in Go. I used your code, first sending an HTTP request with headers for an arbitrary POST body. With 17KB payloads

  • probably half or more of the times where reported as 1ms
  • a few zero ms
  • mostly single digits otherwise
  • but a few as high as the thirties

So the question then is: did the data arrive correctly, even if some parts where slower?

BTW, the retry part

is valid only if sent is zero. If it partially goes through -- which may only be possible in particular implementations and how data is buffered and chunked to head out -- then you'd want to send the remaining part of the "packet" (not to be confused with TCP packets).

3

the length is always 2048 bytes. i split this into packages of 1024 bytes and send them. I use python to receive the data and made it write the amount of bytes it receives. it always receives 1024 bytes. iam new to this so iam not sure how to check if everything gets received.
i was thinking maybe i could set a timer, if it is not send within 10ms cancel and retry.
i adjusted the if (sent < packetSize) since packetsize should always be 1024 this should be better

4

TCP is reliable through things like acknowledgement and retransmission. By your own admission, maybe you shouldn't try to outsmart it. Even setting TCP_NODELAY seems like a premature optimization.

How do you know the hiccups you (and I) see are not the protocol doing exactly what it is supposed to do, and handling some transient issue?

How are you going to cancel the call? write is blocking. Do you know how to cancel that guarantees the packet won't get transmitted? If not--

If you retry, is the receiver able to distinguish that two identical payloads are not coincidentally the same; versus an original, that was not successfully cancelled, and a retry?

I suggest working at a higher level and consider devising a "checksum" (i.e. more sophisticated than just adding a bunch of stuff together) to verify the data is received correctly. And sure, monitor the performance to see if there are any trends and to catch real outliers.

5

One thing which may be causing the timing differences is the PC and operating system you are using.
Most operating systems are focused on the user's operations, and not the network. So, you can try to shut down as many 'services' as possible on your PC. Shut down as much hardware as you can: Bluetooth, WiFi, anything else. Also make sure no additional network traffic is passing on your small space of ethernet.