Send multiple messages through Eth shield

Projects Programming
1

Hi all,

I'm trying to send multiple messages from my Arduino to a server.
In the main loop I have a for cycle that tries to send four (identical) messages, but only the first is sended because the client.connect() method returns false after the first message.

Note that I wrote my code using the example given in a previous post:
http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1254745295

The Arduino code:

#include <SPI.h>
#include <Ethernet.h>

byte mac[] = { 0x90, 0xA2, 0xDA, 0x00, 0x20, 0x45 };
byte serverIp[] = { 192, 168, 33, 17 };
byte clientIp[] = { 192, 168, 33, 16 };

Server server(10001);
Client client2(clientIp, 10002);


void setup()
{
  Serial.begin(57600);
  Ethernet.begin(mac, serverIp);
  server.begin();
  Serial.println("Server started");
}


void loop()
{
  
  // listen for incoming clients
  Client client = server.available();
  
  if (client) {
    Serial.println("Client available");
    
    int i = 0, len = 0;
    byte hbuf[22];
    byte pbuf[256];
    byte respHeader[] = { 0x32, 0x4F, 0x44, 0x32, 0x10, 0x9, 0x3E, 
                          0x7E, 0x0,  0x0,  0x0,  0x30, 0x20, 0x10, 
                          0x40, 0x50, 0x21, 0x0,  0x0,  0x51, 0x22, 
                          0x32 
                        };
    byte respPayload[] = { 0x42, 0x43, 0x44, 0x45, 0x46, 0x47, 0x48, 
                           0x49, 0x50, 0x20, 0x21, 0x22, 0x23, 0x24, 
                           0x25, 0x26, 0x27, 0x28, 0x29, 0x30, 0x9, 
                           0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 
                           0x17, 0x18, 0x19, 0x1,  0x2,  0x3,  0x4, 
                           0x5,  0x6,  0x7,  0x8,  0x20, 0x21, 0x22, 
                           0x23, 0x24, 0x25, 0x26, 0x27, 0x28, 0x29, 
                           0x30 
                         };
    
    // lettura header
    Serial.println("Reading header");
    while (client.connected() && i < 22) {
      if (client.available()) {
        hbuf[i] = client.read();
        i++;
      }
    }
    
    if (i != 22) {
      Serial.println("Insufficient bytes for parsing");
    }
    
    Serial.print("Header bytes: ");
    for (i = 0; i < 22; i++) {
      Serial.print(hbuf[i], HEX);
      Serial.print(" ");
    }
    Serial.println();

    i = 0;
    len = hbuf[0];  // first header's byte
    Serial.println("Reading payload");
    while (client.connected() && i < len) {
      if (client.available()) {
        pbuf[i] = client.read();
        i++;
      }
    }
    
    if (i != len) {
      Serial.println("Insufficient bytes for parsing");
    }
    
    Serial.print("Payload bytes: ");
    for (i = 0; i < len; i++) {
      Serial.print(pbuf[i], HEX);
      Serial.print(" ");
    }
    Serial.println();
    
    for (i = 0; i < 4; i++) {
      sendResponseMessage(respHeader, respPayload);
    }
    
    delay(1);
    client.stop();
  }
  
}


void sendResponseMessage (byte *respHeader, byte *respPayload)
{
  int i, len = 0;
  
  if (client2.connect()) {
    
    Serial.println("Connected");
    Serial.println("Sending response message...");
    
    if (respHeader) {
      Serial.println("Sending the message header...");
      for (i = 0; i < 22; i++) {
        client2.write(* (respHeader + i));
      }
      Serial.println("Header sended");
    }
    
    len = respHeader[0];
    if (respPayload) {
      Serial.println("Sending the message payload...");
      for (i = 0; i < len; i++) {
        client2.write(*(respPayload + i));
      }
      Serial.println("Payload sended");
    }
  } else {  // connection failed
    Serial.println("Connection failed");
  }
  
  delay(300);
  client2.flush();
  
  if (!client2.connected()) {  // no more connected
    Serial.println();
    Serial.println("Disconnecting.");
    client2.stop();
//    for(;;)
//	;
  }
  else {  // disconnect anyway
    client2.stop();
  }

}

And this is the output:

Server started
Client available
Reading header
Header bytes: 32 4F 44 32 10 9 3E 7E 23 38 11 30 20 10 40 50 21 31 41 51 22 32 
Reading payload
Payload bytes: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 

Connected
Sending response message...
Sending the message header...
Header sended
Sending the message payload...
Payload sended

disconnecting.
Connection failed

disconnecting.
Connection failed

disconnecting.
Connection failed

disconnecting.
2 
  client2.flush();

Why?

If the client generated a response (like, maybe a clue as to the problem), just silently ignore it. Seems to me that it would be a lot better to read and print the response.

3

I deleted the client2.flush() statement, but the result is the same.

Seems to me that it would be a lot better to read and print the response.

I don't understand this.

4

Do you like knock, knock jokes?

Knock, knock.
Who's there"
// The rest of the joke

Well, you went "knock, knock" to the client. The client responded "who's there". You did not listen to that response.

After the first message is sent, you wait a little while, and then send again. If the client responded "No comprende, si vous plait. Goodbye", you respond by trying to send again.

It seems to me that you should call client.available() and client.read(), in a while loop, to see if the client did say anything that might, just possibly, explain why the next call fails.

5

I'm playing another kind of joke.

The first guy (server) says: "Give me all your soccer stickers, one by one"
The second guy (client) says: "Here's Diego Armando Maradona, and here's Roberto Baggio, and..."

To better explain the data flow:

  • the Arduino "listen" for a request;
  • when the request arrives, it sends collected data (custom byte arrays, in this case) more than once;
  • the server "listen" for each "stickers", then collect and process it, without giving feedback to the Arduino;

But...
As previously said, the Client.connect() returns false after the first connect-write-stop flow.

If, for each message sended, I close the socket from the other end of the connection, all works fine.

But why Client.connect() return false if I stopped and I "maintain" the server side open?

6

the server "listen" for each "stickers", then collect and process it, without giving feedback to the Arduino;

Without knowing what server you are talking to, we can't assume that the server is not giving any feedback.

It might be true. It might not.

if(client.available() > 0)
{
   while(client.available() > 0)
   {
      Serial.print(client.read());
   }
}

would remove any ambiguity.

I think from your original code that the client is doing something more like:
Here's Diego Armando Maradona, and here's Diego Armando Maradona, and here's Diego Armando Maradona, and here's Diego Armando Maradona.

I guess I don't understand why you need to send the same data 4 times.

But why Client.connect() return false if I stopped and I "maintain" the server side open?

Perhaps, and this is strictly a guess, you should not be closing the client connection if the server is (trying to) maintain an open connection.

7

PaulS:

I think from your original code that the client is doing something more like:
Here's Diego Armando Maradona, and here's Diego Armando Maradona, and here's Diego Armando Maradona, and here's Diego Armando Maradona.

I guess I don't understand why you need to send the same data 4 times.

Because this is only an example to show you my connection problem.

8

PaulS:

the server "listen" for each "stickers", then collect and process it, without giving feedback to the Arduino;

Without knowing what server you are talking to, we can't assume that the server is not giving any feedback.

It might be true. It might not.

I wrote the server, so I know how it works.
Here it is, if it can help:

package examples.send.multiple.mini;

import java.io.IOException;
import java.io.InputStream;
import java.net.ServerSocket;
import java.net.Socket;

public class Example2 {

  public static void main(String[] args) {
    
    ServerSocket server = null;
    
    if (server == null) {
      try {
        server = new ServerSocket(10002);
        System.out.println("Server started");
      }
      catch (IOException e) {
        System.out.println("Error while creating the server");
        e.printStackTrace();
      }
    }
    receiveMessages(server);
  }
  
  
  private static void receiveMessages(ServerSocket server) {
    byte[] receivedHeaderBytes = null, receivedMessageBytes = null;
    byte[] totalReceivedBytes = null;
    int i;
    
    /*
     * Receiving multiple responses
     */
    for (i = 0; i < 2; i++) {
      System.out.println("Receiving the message num. " + (i + 1));
      totalReceivedBytes = receiveMessage(server);
      
      if (totalReceivedBytes == null || totalReceivedBytes.length == 0) {
        System.out.println("Error while receiving the response message");
        return;
      }
      
      receivedHeaderBytes = new byte[22];
      receivedMessageBytes = new byte[totalReceivedBytes.length - 
                                      receivedHeaderBytes.length];
      
      System.arraycopy(totalReceivedBytes, 0, receivedHeaderBytes, 0, 
          receivedHeaderBytes.length);
      System.arraycopy(totalReceivedBytes, receivedHeaderBytes.length, 
          receivedMessageBytes, 0, receivedMessageBytes.length);
      
      // print the received byte arrays
      System.out.print("Header bytes: ");
      for (i = 0; i < receivedHeaderBytes.length; i++) {
        System.out.printf("%02X ", receivedHeaderBytes[i]);
      }
      System.out.println();
      System.out.print("Message bytes: ");
      for (i = 0; i < receivedMessageBytes.length; i++) {
          System.out.printf("%02X ", receivedMessageBytes[i]);
        }
      System.out.println();
      
    } // end for
  }
  
  private static byte[] receiveMessage(ServerSocket server) {
    Socket client = null;
    InputStream sockInput = null;
    byte[] buf = null, retBuf = null;
    int bytesRead = 0;
    
    try {
      client = server.accept();
      System.out.println("Client connected");
      
      sockInput = client.getInputStream();
      buf = new byte[256];
      
      Thread.sleep(200);
      bytesRead = sockInput.read(buf, 0, buf.length);
      System.out.println("Input stream received");
      
      if (bytesRead < 0) {
        System.out.println("Data reading failed");
        return null;
      }
      System.out.println("Readed " + bytesRead + " bytes");

      // build a new buffer containing only the received bytes
      retBuf = new byte[bytesRead];
      System.arraycopy(buf, 0, retBuf, 0, bytesRead);
      
    }
    catch (IOException io) {
      io.printStackTrace();
    }
    catch (InterruptedException ie) {
      ie.printStackTrace();
    }
    return retBuf;
  }
}
9

ramo102:

PaulS:

I think from your original code that the client is doing something more like:
Here's Diego Armando Maradona, and here's Diego Armando Maradona, and here's Diego Armando Maradona, and here's Diego Armando Maradona.

I guess I don't understand why you need to send the same data 4 times.

Because this is only an example to show you my connection problem.

And because repeated stickers happen. :slight_smile:

10

I wrote the server, so I know how it works.

How did you test your server to verify it is working as expected?

11

It is my fault. :blush:

I re-write the server:

package examples.send.multiple.ultimate;

import java.io.BufferedInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.net.ServerSocket;
import java.net.Socket;

public class MessageServer {

  private static final int HEADER_LEN = 22;
  private static final int MESSAGE_LEN = 256;
  private static final int PORT = 10002;
  private static final int THREAD_SLEEP = 33;   // arduino, 255 bytes payload

  private static boolean connected = false;
  private static ServerSocket server = null;

  public static void main(String[] args) {
    Socket client = null;
    InputStream is = null;
    BufferedInputStream bs = null;
    long currentTime = 0, elapsedTime = 0;

    if (server == null) {
      try {
        server = new ServerSocket(PORT);
        System.out.println("Server started");

        client = server.accept();
        connected = true;
        System.out.println("Client connected");
        is = client.getInputStream();
        System.out.println("Input stream obtained");
        bs = new BufferedInputStream(is);
        System.out.println("Buffered input reader obtained\n");

        /*
         * Listen for an unknown number of messages
         */
        System.out.println("Receiving messages...");
        currentTime = System.currentTimeMillis();

        receiveMessages(client, bs);

        elapsedTime = System.currentTimeMillis() - currentTime;
        System.out.println("\nReceiving messages finished\n");
        System.out.println("Messages received in " + elapsedTime + " msecs");

        System.out.println("Closing the input buffer stream");
        bs.close();
        System.out.println("Closing client connection");
        client.close();
        connected = false;
        System.out.println("Closing server connection");
        server.close();
      }
      catch (IOException e) {
        System.out.println("Error while creating the server");
        e.printStackTrace();
      }
    }
  }

  private static void receiveMessages(Socket client, BufferedInputStream input) {

    byte[] receivedHeaderBytes = null, receivedMessageBytes = null;
    byte[] totalReceivedBytes = null;
    int numMessages;
    int i;

    /*
     * Receiving multiple responses
     */
    numMessages = 0;
//    long startingTime = System.currentTimeMillis();

    while (connected == true) {
      System.out.println("\nReceiving message...");
      totalReceivedBytes = receiveMessage(input);
      
      if (totalReceivedBytes == null || totalReceivedBytes.length == 0) {
        System.out.println("Error while receiving the response message");
        return;
      } else if (totalReceivedBytes.length == 1 || totalReceivedBytes.length == 2) {
        System.out.println("No sufficient bytes to parse a new message");
        return;
      }
      System.out.println("Received message number: " + (numMessages + 1));
      numMessages++;

      receivedHeaderBytes = new byte[HEADER_LEN];
      receivedMessageBytes = new byte[totalReceivedBytes.length - 
                                      receivedHeaderBytes.length];

      System.arraycopy(totalReceivedBytes, 0, receivedHeaderBytes, 0, 
          receivedHeaderBytes.length);
      System.arraycopy(totalReceivedBytes, receivedHeaderBytes.length, 
          receivedMessageBytes, 0, receivedMessageBytes.length);

      // print the received byte arrays
      System.out.print("Header bytes: ");
      for (i = 0; i < receivedHeaderBytes.length; i++) {
        System.out.printf("%02X ", receivedHeaderBytes[i]);
      }
      System.out.println();
      System.out.print("Message bytes: ");
      for (i = 0; i < receivedMessageBytes.length; i++) {
          System.out.printf("%02X ", receivedMessageBytes[i]);
        }
      System.out.println();

    } // end for

  }
  
  private static byte[] receiveMessage(BufferedInputStream input) {

    byte[] buf = null, retBuf = null;
    int bytesRead = 0, len = 0;

    try {
      buf = new byte[MESSAGE_LEN];

      // read the message header first
      Thread.sleep(THREAD_SLEEP);
      bytesRead = input.read(buf, 0, HEADER_LEN);

      if (bytesRead < 0) {
        System.out.println("Insufficient bytes to parse header");
        return new byte[1];
      }
      System.out.println("Header bytes received");
      System.out.println("Readed " + bytesRead + " bytes");

      // store the payload length (fix the "signed byte java problem"
      len = buf[0] & 0xFF;

      // check the payload length (header's first byte)
      if (len <= 0) {
        System.out.println("Invalid payload length");
        return null;
      }

      // create the return buffer
      retBuf = new byte[HEADER_LEN + len];

      // copy the header to the return buffer
      System.arraycopy(buf, 0, retBuf, 0, HEADER_LEN);

      Thread.sleep(THREAD_SLEEP);
      bytesRead = input.read(buf, 0, len);

      if (bytesRead < 0) {
        System.out.println("Insufficient bytes to parse message");
        return new byte[2];
      }
      System.out.println("Payload bytes received");
      System.out.println("Readed " + bytesRead + " bytes");

      if (len > (retBuf.length - HEADER_LEN)) {
        System.out.println("The payload exceed the message maximum length!");
        return null;
      }

      // copy the payload to the return buffer
      System.arraycopy(buf, 0, retBuf, HEADER_LEN, len);
    }
    catch (IOException e) {
      e.printStackTrace();
    }
    catch (InterruptedException ie) {
      ie.printStackTrace();
    }

    return retBuf;
  }
}

And the Arduino part too:

#include <SPI.h>
#include <Ethernet.h>

const int PORT = 10002;
const int MESSAGE_NUM = 100;
const int HEADER_LEN = 22;
const int MAX_PAYLOAD_LEN = 256;

byte mac[] = { 0x90, 0xA2, 0xDA, 0x00, 0x20, 0x45 };
byte serverIp[] = { 192, 168, 0, 2 };
byte clientIp[] = { 192, 168, 0, 1 };

Client client(clientIp, PORT);

void setup()
{
  Serial.begin(57600);
  Ethernet.begin(mac, serverIp);
}

void loop()
{
  int i = 0, len = 0;

  // header to send (22 bytes, fixed)
  // the first byte is the payload length
  byte respHeader[] = { 0xFF, //0xC8, //0x96, //0x64, //0x32, 
    0x4F, 0x44, 0x32, 0x10, 0x9, 0x3E, 
    0x7E, 0x0,  0x0,  0x0,  0x30, 0x20, 0x10, 
    0x40, 0x50, 0x21, 0x0,  0x0,  0x51, 0x22, 
    0x32 
  };
  
  // message to send (variable length)
  byte respPayload[] = {
    0x1, 0x2, 0x3, 0x4, 0x5, 0x6, 0x7, 0x8, 0x9, 0x10, 
    0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18, 0x19, 0x20, 
    0x21, 0x22, 0x23, 0x24, 0x25, 0x26, 0x27, 0x28, 0x29, 0x30, 
    0x31, 0x32, 0x33, 0x34, 0x35, 0x36, 0x37, 0x38, 0x39, 0x40, 
    0x41, 0x42, 0x43, 0x44, 0x45, 0x46, 0x47, 0x48, 0x49, 0x50, 
    0x51, 0x52, 0x53, 0x54, 0x55, 0x56, 0x57, 0x58, 0x59, 0x60, 
    0x61, 0x62, 0x63, 0x64, 0x65, 0x66, 0x67, 0x68, 0x69, 0x70, 
    // 70
    0x1, 0x2, 0x3, 0x4, 0x5, 0x6, 0x7, 0x8, 0x9, 0x10, 
    0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18, 0x19, 0x20, 
    0x21, 0x22, 0x23, 0x24, 0x25, 0x26, 0x27, 0x28, 0x29, 0x30, 
    0x31, 0x32, 0x33, 0x34, 0x35, 0x36, 0x37, 0x38, 0x39, 0x40, 
    0x41, 0x42, 0x43, 0x44, 0x45, 0x46, 0x47, 0x48, 0x49, 0x50, 
    0x51, 0x52, 0x53, 0x54, 0x55, 0x56, 0x57, 0x58, 0x59, 0x60, 
    0x61, 0x62, 0x63, 0x64, 0x65, 0x66, 0x67, 0x68, 0x69, 0x70, 
    // 140
    0x1, 0x2, 0x3, 0x4, 0x5, 0x6, 0x7, 0x8, 0x9, 0x10, 
    0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18, 0x19, 0x20, 
    0x21, 0x22, 0x23, 0x24, 0x25, 0x26, 0x27, 0x28, 0x29, 0x30, 
    0x31, 0x32, 0x33, 0x34, 0x35, 0x36, 0x37, 0x38, 0x39, 0x40, 
    0x41, 0x42, 0x43, 0x44, 0x45, 0x46, 0x47, 0x48, 0x49, 0x50, 
    0x51, 0x52, 0x53, 0x54, 0x55, 0x56, 0x57, 0x58, 0x59, 0x60, 
    // 200
    0x1, 0x2, 0x3, 0x4, 0x5, 0x6, 0x7, 0x8, 0x9, 0x10, 
    0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18, 0x19, 0x20, 
    0x21, 0x22, 0x23, 0x24, 0x25, 0x26, 0x27, 0x28, 0x29, 0x30, 
    0x31, 0x32, 0x33, 0x34, 0x35, 0x36, 0x37, 0x38, 0x39, 0x40, 
    0x41, 0x42, 0x43, 0x44, 0x45, 0x46, 0x47, 0x48, 0x49, 0x50, 
    // 250
    0x1, 0x2, 0x3, 0x4, 0x5
  };
  
  if (client.connect()) {

    for (i = 0; i < MESSAGE_NUM; i++) {
      sendResponseMessage(respHeader, respPayload);
    }

    delay(1);
    client.stop();

  } else {  // not connected
    Serial.println("Connection failed");
  }
}

void sendResponseMessage (byte *respHeader, byte *respPayload)
{
  int i, len = 0;
  boolean isConnected = false;
  
  if (respHeader) {
    Serial.println("Sending the message header...");
    for (i = 0; i < 22; i++) {
      client.write(* (respHeader + i));
    }
    Serial.println("Header sended");
  }

  len = respHeader[0];

  if (len <= 0) {
    Serial.println("Invalid message length");
    return;
  }

  if (respPayload) {
    Serial.println("Sending the message payload...");
    for (i = 0; i < len; i++) {
      client.write(*(respPayload + i));
    }
    Serial.println("Payload sended");
  }
}

Now I can send multiple messages. I tried sending up to 200 messages, and it works. :slight_smile:

12

I have another question now.

I tried this scenario (client that sends a lot of message to the server, using the same connection) also with a java client.
In this case the communication is very fast, because the server is able to read from the input buffer without using any delay.

When using the Arduino client, instead, I have to insert delays before each read from the input buffer (as you can see from the code above). So while in the former case I can process 100 messages in less than one second, in the latter one the server spends more than three seconds.

Is there a way to speed up the Ethernet communication?
(and do I have to ask this within a different thread?)

13

When using the Arduino client, instead, I have to insert delays before each read from the input buffer (as you can see from the code above).

I guess I can't. I don't see that the Arduino is reading any response from the server. I see that any response from the server is simply flushed.

14

"...before each read from the input buffer..."
I mean the server's input buffer here.

The client, infact, does not receive anything.

15

So, when a Java app is the client (you have a strange sense of the relationship between client and server), the server doesn't need to delay before reading. But, when the Arduino is the client, you do?

If that is really the case, the problem is with the server making unrealistic expectations about the incoming data stream (speed or availability of data).

16

PaulS:
(you have a strange sense of the relationship between client and server)

Yes, I know. It's a long story...

PaulS:
If that is really the case, the problem is with the server making unrealistic expectations about the incoming data stream (speed or availability of data).

Probably.
I counted the time spended from the Arduino side to send 200 bytes through the Ethernet shield, and the result is 52 msecs (more or less).
So it seems that the data are available within a little amount of time.