More reliable serialcommunication

Let me first start off by saying that I am aware that this question HAS been asked a number of times here, so please spare me. Here is my code

#define BUFFSIZE 300
void setup() 
{
  Serial.begin(115200);
}

void loop() 
{ 
  int multiplier=1;
  char* buf = (char*) malloc(BUFFSIZE);
  int count = 0;
  int terminated = 0;
  while(!terminated) 
  {
    if(!Serial.available())continue;
      if(count>=BUFFSIZE*multiplier)
      {
        multiplier++;
        buf = (char*)realloc(buf,BUFFSIZE*multiplier);     
      }
      buf[count] = Serial.read();
      if(buf[count]=='\n'){terminated=1;}
      count++;
  }
  buf[count]='\0';
  if(count!=0)
  {
  Serial.write("received: ");
  Serial.write(buf);
  Serial.flush();
  Serial.write("\n");
  Serial.write("END...");
  Serial.write("\n");   
}
  free(buf);
  delay(100);
}

,
When I used to send data under 1.5k (cmd message really, terminated by ‘\n’), the code seems to echo back fine. However, I have read on a number of threads that this method is unreliable (even if arduino has enough ram). I have been told that I have to implement handshaking to handle error and loss. Is there no standard library to do this? Most of the answers I have found here are just roll-out-your-own partial solutions. I mean it has to be right? The task is most mundane but somehow I cannot find it.

Moderator edit: [code][/code] tags added. (Nick Gammon)

Have a look at the examples in serial input basics. You can easily increase the array size.

If I was using a large percentage of total ram I certainly would not use malloc() or its counterparts - just create a fixed array big enough for the largest data.

…R

Why on earth are you malloc’ing a buffer? Just make it a static variable. RAM is RAM however you use it.

And realloc? Why?

Read this: Gammon Forum : Electronics : Microprocessors : How to process incoming serial data without blocking

What is this buffer? If it is data (like a string of numbers) you don’t even need to store it all, just decode it as it is received. My link above shows how to to that.

You can implement error-detection if you want. Send some sort of sumcheck with the data and check it upon reception.

Is this between two Arduinos, or an Arduino and a PC?

If it is between two Arduinos, I wrote a library a while back which does error detection:

http://www.gammon.com.au/forum/?id=11428

It might be labelled “RS485” but since the RS485 on that page goes out the serial port, it is the same thing.

The communication is between Arduino and PC. The thing is, I want to allow sender (PC) to send a variable length message to the Arduino so I think it would be appropriate to use malloc and realloc (as opposed to using, say static 1800b static buffer). Is there something inherently bad with dynamic allocation that I am not aware of?
Or is it because ram unused = ram wasted
The reason I am buffering it because I plan my Arduino to, in a loop

(1)wait indefinitely and buffer all data from pc until '\n' is reached (call this data in buffer msg)
(2)msg is checked against list of valid command
(3)Arduino either goes off on a branch to execute that command and return "exit" code to PC or reject the input (kind of like sending back NACK)

I surely can implement my checksum, my intuition tells me that I should break the msg into pkgs and use something like datagram with header containing sequence number.Would that be viable?

Is there something inherently bad with dynamic allocation that I am not aware of?

Yes. It is called heap fragmentation. In particular, allocating memory piecemeal (eg. with realloc) is exactly the thing likely to cause it. You can tolerate a certain maximum message size (due to the small amount of RAM you have). Why not just use a fixed buffer?

realloc() will try to simply expand the memory it has (by growing it). If that isn't possible it has to allocate a new amount, and then copy. So for example, if you are up to 900 bytes, and it can't just grow it to 1000 bytes, it has to allocate 1000 (plus the original 900) and then copy and free up the first 900. You can see you will quickly run out of RAM at that rate.

I surely can implement my checksum, my intuition tells me that I should break the msg into pkgs and use something like a udp datagram.Would that be viable?

Certainly, which is exactly what my library does.

The thing is, I want to allow sender (PC) to send a variable length message to the Arduino so I think it would be appropriate to use malloc and realloc (as opposed to using, say static 1800b static buffer).

Why? What purpose is served by:

  1. sending so much character data
  2. fragmenting memory be using malloc and realloc?