Why am I missing last 16 bytes with serial communication?

Using Arduino Programming Questions
#1

Hello to all,

I have function that initates serial communication with another MCU, then listens and should collect a 16kb file send from the other end and write it in memory. My problem is that the last 16 bytes are missing - one for each loop that I make, but not inbetween loops, but at the end. Here is my code:

void inbound_transfer() {
  String filename = "/backup.bin";
  if (SPIFFS.exists(filename)) {
    SPIFFS.remove(filename);
  }
  File f = SPIFFS.open(filename, "w");
  size_t fsize = f.size();
  const byte numChars = 512;
  String receivedConfirmation;
  byte rc;
  bool moredata = 1; //there are bytes in the file that need to be received
  //prepare sender
  Serial.print("snd");
  delay(10);
  while (moredata == 1) {

    // READ ANSWER FROM REMOTE DEVICE
    byte buff[1024];
    int len =  readByteFromSerial(buff, 1024);
    if (len > 0) {
      len = 0;
      f.write(buff, 1023);
    }
    if (Serial.available() > 0) {
      moredata = 1;
      yield();
    }
    else moredata = 0;
  }
  f.close();
  transferCommand = 0;
}

and the readByteFromSerial function :

int readByteFromSerial(byte *buffer, int max_len )
{
  int pos = 0;
  unsigned long int init_time = millis();
  unsigned long int wait_time = init_time + 1000;
  while (!Serial.available())
  {
    //    Serial.println("enter waiting for serial");
    if (millis() >= wait_time) {
      // no responce - just leave
      // Serial.println("time is up");
      return pos;
    }
  }

  while (pos < max_len - 1)
    //for ( ; pos <= max_len - 1; pos++)
  {
    if (Serial.available() > 0) {
      byte readch = Serial.read();
      buffer[pos] = readch;
      pos++;
    }
  }
  return pos;
}
#2

Are any Arduinos being used? Post the whole of your code and post it in code tags!

Mark

#3

The function readByteFromSerial is unknown (No results found for readByteFromSerial site:https://www.arduino.cc/en/reference), so it might contain the error, but this looks sus:

    byte buff[1024];
    int len =  readByteFromSerial(buff, 1024);
    if (len > 0) {
      len = 0;
      f.write(buff, 1023)

Read up to 1024 bytes, write up to 1023...

Pogo

#4

You call f.write with 1023, instead of with len, so its going to miss every 1024'th character and write lots of junk bytes at the end of the file.

The last 16 bytes thing is probably at the other end of the serial connection where something is reseting without waiting for the serial buffer to drain.

#5 
while (pos < max_len - 1)

pos last position is max_len - 2. It doenst go to the last position in buffer[max_len]

#6

Speklap:

while (pos < max_len - 1)

pos last position is max_len - 2. It doenst go to the last position in buffer[max_len]

That was it!

I changed it to

while (pos < max_len)
#7

That's the beauty of zero indexed arrays. No more +/-1's

#8

Speklap: That's the beauty of zero indexed arrays. No more +/-1's

Can you please share a link with more theoretical info or examples about "zero indexed arrays".

#9

Zero indexed arrays are arrays which first element is the zeroth element. You start counting with zero.

int length = 5;
int arr[length];

arr[0] = 3;         //*arr = 3;
arr[1] = 4;         //*(arr+1) = 4;
//...
arr[length-1] = 1;        //*(arr+length - 1) = 1;

As you can see, arr is a pointer to a memory location where the array starts. So if you want to have the first element, you need the memory location arr with no offset, or in other words, you add zero to it. You want the second element, you add 1 to arr, and you get to the memory location of the next element.

I tried to explain it in my own words, since I have no theoretical info or examples. You start counting with zero, not with one.

#10