lost end information when using EthernetClient.readString()

hi everyone,

I’m new with the Arduino. I want to extract a JSON outof a response of a SQL server. I use EthernetClient.readString() to read the received data from the buffer. But it seems to lost the end part of the data. I also tried cleint.readString(’\0’) and set a larger timeout. It still doesn’t work.

The fully response that should be contained in String c is:

HTTP/1.1 200 OK
Server: Apache
Vary: Accept-Encoding,User-Agent
Transfer-Encoding: chunked
Content-Type: text/html; charset=UTF-8

(some debug information, like username and posted data are listed here)
??MYSQL??[{“varname”:“n”,“dimRows”:“1”,“dimCols”:“1”,“unit”:“1”,“data”:“INT”,“value”:“10”}]
0
disconnecting,

but instead it prints:

HTTP/1.1 200 OK
Server: Apache
Vary: Accept-Encoding,User-Agent
Transfer-Encoding: chunked
Content-Type: text/html; charset=UTF-8

(some debug information, like username and posted data are listed here)
??MYSQL??[{“varname”:“n”,“dimRows”:“1”,“dimCols”:“1”,“unit”:"
disconnecting,

The end part is missing. I can only get the full response by using client.read() function. but it returns no String, and can’t be parsed with the String member function.

I’m really appreciated if there’s someone helps me out.

This is my code:

#include<SPI.h>
#include<Ethernet.h>
#include<ArduinoJson.h>
#define buffersize 2000

byte mac={ 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
char server=SSID;
IPAddress ip(xxx, xxx, x, xxx);

// DATA
String GetINT=“user=user1&pass=123&variabletyp=int&variable2=2”;

EthernetClient client;

bool post(String data)
{
if(client.connect(server,80))
{
Serial.println(“connected”);

client.println(“POST /server.php HTTP/1.1”);
client.println(“Host: SSID”);
client.println(“User-Agent: Arduino/1.0”);
client.println(“Connection: colse”);
client.println(“Content-Type: application/x-www-form-urlencoded”);
client.print("Content-Length: ");

client.println(data.length());
client.println();
client.println(data); // Post login data
client.println();
delay(200);
}
else{Serial.println(“connected failed”);}
}

void setup()
{
Serial.begin(115200);
while(!Serial){}
if(Ethernet.begin(mac)==0)
{Serial.println(“Failed to configure Ethernet using DHCP”);
Ethernet.begin(mac,ip);}
delay(1000);
Serial.println(“connecting…”);
client.setTimeout(5000);

post(GetINT);

}

void loop()
{ String s;
if(client.available())
{ String c=client.readStringUntil(’\0’);
Serial.print(c);
s=c.indexOf("??**");
Serial.print(s);

}

if(!client.connected())
{
Serial.println();
Serial.println(“disconnecting,”);
client.stop();
while(true);
}
}

The end part is missing. I can only get the full response by using client.read() function. but it returns no String, and can't be parsed with the String member function.

That should be good reasons to drop this idea of using the String class that will poke holes in your memory. Build a buffer with read and you can parse with standard C functions

I wonder if you are getting hit by the default 1000 millisecond timeout. Try using client.setTimeout(2000); to see if that reads more characters.

Another choice is to use '\n' as the terminator and check each line to see if (c.startsWith("??MYSQL??")).

String c=client.readStringUntil('\0');

The sender is NEVER going to put a NULL on the serial port, if it is writing ASCII data (as your server obviously is).

If you have ANY control over the output, make it output some clearly recognizable end of packet marker that is not going to appear in the packet.

If you don't have any control, use a different approach where you DO have control.

J-M-L:
That should be good reasons to drop this idea of using the String class that will poke holes in your memory. Build a buffer with read and you can parse with standard C functions

Thanks for reply! I have defined a buffer in the beginning, just don’t know how it exactly works and with which function to read the Respons at the first step. I have also searched other ways to read data, like with cleint.read() which returns the complete response. But i don’t know which c function I can use to parse the JSON string. Could you pls name some functions, I can try myself.

Thanks very much!

johnwasser:
I wonder if you are getting hit by the default 1000 millisecond timeout. Try using client.setTimeout(2000); to see if that reads more characters.

I also tried with client.setTimeout(2000~5000), they give the same results, no more characters is readed out. The response is still incomplete, and have no idea about it.

johnwasser:
Another choice is to use '\n' as the terminator and check each line to see if (c.startsWith("??MYSQL??")).

still working on it.
Thanks for reply!

PaulS:

String c=client.readStringUntil('\0');

The sender is NEVER going to put a NULL on the serial port, if it is writing ASCII data (as your server obviously is).

If you have ANY control over the output, make it output some clearly recognizable end of packet marker that is not going to appear in the packet.

If you don't have any control, use a different approach where you DO have control.

Actually that is one of my problems. I am really not clear about what kind of data I'm getting received from the server and how it ends. In fully responded cases:

PaulS:
HTTP/1.1 200 OK
Server: Apache
Vary: Accept-Encoding,User-Agent
Transfer-Encoding: chunked
Content-Type: text/html; charset=UTF-8

(some debug information, like username and posted data are listed here)
??MYSQL??[{"varname":"n","dimRows":"1","dimCols":"1","unit":"1","data":"INT","value":"10"}]
0
disconnecting,

Could you tell me, where I can know that the server is writing ASCII data? and which function or class I can use to parse it?
The JSON string is after the trigger "??MYSQL??", I supposed to recognize this trigger and cut out the string behind it.
Thanks very much!

Why don’t you just do a while loop and print out byte by byte what is being received ? Then you’ll be better equipped to parse that data

The JSON string is after the trigger "??MYSQL??", I supposed to recognize this trigger and cut out the string behind it.

The "trigger" "??MYSQL??" is useless. It adds NO value to the server output. The data of interest starts with the '[' and ends with the ']'. Just store the data, after the '[' arrives, and stop storing the data, and parse the data, when the ']' arrives.

The ArduinoJSON library can handle the parsing of the JSON data.