ReadStringUntil does not read until or String just looses it?

Using Arduino Programming Questions
#1

I have a peculiar issue with serial communication and/or strings. I made a small gadget that reads RFID badge, sends logging info to server with ESP8266 and then retrieves a log of previous reads to be displayed on a 1" OLED screen. Board is Pro Micro, Arduino 1.6.6. Everything works fine and dandy except reading the log back from net. I only get part of the data which seems to be quite random length from the start. Sometimes I get all, sometimes just 20% or so, and everything in between.

On server side I have made it simple for Arduino and send the log like this:

{08.11. 23:26 (92231460)#08.11. 23:09 (92231460)#08.11. 23:09 Sininen#08.11. 23:09 Valkoinen#08.11. 23:08 (92231460)#}

‘{’ to tell the response begins, then timestamp + card ID + ‘#’ for each log entry and ‘}’ to tell the response ends.

I have a serial adapter tapped between ESP8266 and Micro Serial port to listen to what ESP8266 is sending back to Arduino. Response for the GET call looks like this:

+IPD,209:HTTP/1.1 200 OK
Date: Sun, 08 Nov 2015 21:26:09 GMT
Server: Apache/2.4.10 (Unix) OpenSSL/1.0.1e-fips mod_bwlimited/1.4
X-Powered-By: PHP/5.4.37
Transfer-Encoding: chunked
Content-Type: text/html

1
{

+IPD,75:45
08.11. 23:26 (92231460)#08.11. 23:09 (92231460)#08.11. 23:09 Sininen#

+IPD,54:30
08.11. 23:09 Valkoinen#08.11. 23:08 (92231460)#}

On Arduino side after making the GET call I read in the response to global String variable responseFromServer with:

Serial1.find("{");
responseFromServer = Serial1.readStringUntil('}');

First reading the Serial1 stream and omitting the content until ‘{’ is found, then reading in stuff to variable until end character ‘}’ is found.

However, what I get in responseFromServer string is this:

+IPD,75:45
08.11. 23:26 (92231460)#08.11. 23:09 (92231460)#08.11. 23:09 S

What is happening? Why is my responseFromServer string truncated? As said, the position it gets truncated is quite random, some times I even get the full response until ‘}’ character.

I have tried my everything to solve the issue but in avail. I build a custom “read until ‘}’ or timeout” method but it works exactly the same: it reads the input stream from Serial1 (ESP8266) for a brief time (definitely not until timeout), exits and in result string I get some random truncated part of the server response.

#2

I can’t see any problems with the code that you have posted but what is going on with the rest of the program is anyone’s guess. A problem with memory seems likely and the use of Strings (uppercase S) as opposed to C style strings (lowercase s) is often regarded as unwise by most members of this forum.

How to posti a programming question

#3

Yeah, that's what I expected about the String... :slight_smile: I should switch to char array?

Your comment about "what is going on with the rest of the program" is suggesting other parts of the program might be messing up the memory or something like that? Other than that it would seem illogical to me if the code elsewhere would mess up the serial reading + storing + printing here, where the reading and printing are consequent operations.

#4

suggesting other parts of the program might be messing up the memory

That is certainly what I had in mind. Who knows, maybe you have declared a huge array of floats for some reason or that you are writing past the end of an arrays somewhere.

#5

Ok, thanks! I certainly don't have huge array of floats or such but I do use Strings excessively... :slight_smile: I think I'll start by experimenting with reserve() for my server result "buffer" or just convert it to fixed length char array. I'll also revise the other Strings and see if I could make do without them.

#6

You may be interested in the examples in Serial Input Basics as an alternative to readStringUntil()

The examples also use strings (small s).

...R

#7

Tried reserve() with my Strings, but it had no effect. Still got truncated Strings in few places every now and then. Then I switched my Serial1 input read buffer to char array and that did it! I’m currently reading the Serial1 one character at a time in a while loop that exits if end character ‘}’ is found or timeout is met. I also reduced the amount of Strings to absolute minimum (without rewriting the code to use char arrays anywhere except the read buffer) and used reserver()'s with sane values for those.

I think my issue was so prominent and showed it self so quickly because I’m running out of memory with the Micro. Memory used reported by IDE is 85%/70%, so not much room for long fragmenting Strings there. :slight_smile: I still try to move some static stuff to PROGMEM if I could squeeze in a frame or two for a fancy animation on the OLED… :wink:

Thanks a lot for the help!

#8