AT-MEGA2560 serial receiver buffer size increase

Projects Programming
1

hi guys i am having a problem with AT-MEGA 2560 serial receiver buffer.

actually my esp32 is sending nested json data to AT-MEGA 2560 via serial2.
now the problem is i am receiving only 80% of my json data , but not 100% of json data.
this is due to serial receiver buffer size of At-MEGA2560.

ESP32 CODE

StaticJsonDocument<1024> doc;
doc["dn"] = "esp32";
doc["d-id"] = "wsp123";
JsonObject metric = doc.createNestedObject("ob1");
metric["mode"]:"on",
JsonObject slot1 = doc.createNestedObject("slot1");
slot1["mode"]:"on";
 slot1["ontime"]:11;
 slot1["offtime"]:12;
JsonObject slot2 = doc.createNestedObject("slot2");
slot2["mode"]:"on";
 slot2["ontime"]:11;
 slot2["offtime"]:12;
JsonObject slot3 = doc.createNestedObject("slot3");
slot3["mode"]:"on";
 slot3["ontime"]:11;
 slot3["offtime"]:12;
JsonObject slot4 = doc.createNestedObject("slot4");
slot4["mode"]:"on";
 slot4["ontime"]:11;
 slot4["offtime"]:12;
serializeJson(doc, Serial2);

my AT-MEGA code

unsigned long SERIAL_WAIT=0;
void loop()
{
    if(milllis()-   SERIAL_WAIT>=10000)
    {
       String str="";
        while(Serial3.available()>0)
        {
             char z=Seial3.read();
            str +=z;
        }
        Serial.println(str);
        SERIAL_WAIT=millis();
    }
}

now i am only getting only 60 - 80% of json data from esp32.
how to resolve it?

2

It's actually due to the way you handle the data coming in....

if you do this

void loop()
{
  if (Serial3.available()>0) Serial.write(Serial3.read()); 
}

do you loose anything ?

I would suggest to study Serial Input Basics to handle this and understand how to handle an asynchronous input.

3

Why do you need a Mega? Esp32 is a way power than Mega and can be used standalone

4

To increase the Serial buffer size, you need to alter core directory of your arduino, which can be implemented safely by following the guide below:

this will create a new board which is compileable with the desired (256 bytes in this case) amount of memory allocated to serial buffer.

5

That will just displace the problem that is created by wrongly handling an asynchronous protocol

The while loop checking available() spins faster than data coming in and thus at some point available becomes 0 and the while loop terminates.

6

Yes I agree. I just provided the information he asked in case rest of the project included long blockages which would hinder his ability to continually check availability

Because in the past I had a similar issue that I solved by increasing buffer size. I also modified the isr routine to inform every 256 byte packet arrival, which was a valid solution in my case with long blockages.

7

i went into the library and increased the serial buffer size from 64 to 512 it worked.

8

i did this also but it did not work. i mean i only received half json message on.
moreover i need to store the incoming message in a string for further processing.

9

it is just a simple project. where i need 50 gpios. i already have a mega with me. so instead of buying a multiplexer i just used what i have. and used esp for wifi purpose.

10

I agree with @J-M-L , your problem is not in the buffer size, it is in the incorrect code handling the incoming messages. Increasing the buffer you are only masking the problem, it will still come out sooner or later

11

why are you checking for input only every 10 seconds? why not read input as soon as it is available and process it when a complete msg(?) is received

void loop()
{
    if (Serial.available ())  {
        while (Serial.available ())
        str += Serial3.read ();

        // what is termination char ??

        Serial.println(str);
    }
}
12

i have studied about serial inputs as you specified.
what i was thinking is
for example it the incoming data is : "apple banana"
now if i use use

void loop()
{
  if (Serial3.available()>0) Serial.write(Serial3.read()); 
}

now if my loop has many functions, then it will print only one byte at a time.
now my other function need the entire string from the serial3, so i used while

while (Serial3.available()>0) Serial.write(Serial3.read());

so this will print all the data that is coming from serial3 and after that it will go to execute next functions.

this is what my understanding in serial input.
if i am doing anything wrong or if there is any better way than please update me.

13

i did not understand ?

14

ok this one is wrong.
so i will change it and as you said i will read it as soon as the incoming message comes.

15

Yes, it is wrong concept.

You never can be sure that your message is received completely. So instead of waiting for receive a 200-300 chars, you must read them by 2-3 chars as quickly as possible and store the text in local variable. The end of the data must be properly terminated to determine when the dta will be ready to process. While you received a terminator you start your text processing.
If you handle serial input this way, you won't need a buffer more than 64 bytes

16

how do you know when a complete msg is received?

17

guys please help me with one situation.

consider there are two devices AT-MEGA and ESP32.

ESP32 has only one job. the job is to send a json message to AT-MEGA every 1 second VIA serial communication.

now coming to AT-MEGA part there are 3 functions in void loop() function.
the first function job is to read from serial3 receiver buffer (where there is esp32 json data).
the second function needs the json data from serial 3 as input.
the third function is unrelated to above both functions. it takes an execution time of 1second.

now what J-M-L Jackson
suggest me is to use

if (Serial3.available()>0) Serial.write(Serial3.read());

but what i am thinking is if i use if() condition then it will read only one 1 character at a time.
where my 2nd function needs the entire json string so i used

while (Serial3.available()>0) Serial.write(Serial3.read());

by using this while loop() it will read entire json data from serial3 so i can use that json data as input to my second function.

here i am doing any thing wrong or any other problem will come i want to know?

next b707
says that my problem is not with the buffer size. i did not understand this one.
so please explain this with my above problem .

18

as i am dealing with json data. if i get half data then

void Appdata(String str)
{
  StaticJsonDocument<1024> ESP;
  deserializeJson(ESP, str);
  DeserializationError error = deserializeJson(ESP, str);
  if (error) 
  {
    Serial.println("DESERIALIZING  APP-ESP32 DATA FAILED");
    return;
  }
}

see this String str will contain the json data from ESP32 via serial.
if i dont get full message it will give error.

19

What is the function? Could it be done by many short periods like 10-20ms rather than an entire work for 1 second?

20

It is not related to the way how you received the json message - at once with a huge buffer or in 2-3 bytes pieces. In any case you should not start processing until you have received the complete message.
In your current program you don't even check whether you received the entire message or not.