Trouble sending large amounts of data over serial

Hello all. I have a robot that I'm using with ROS2 and I need to send IMU data over Serial for it. In total, I have to send 10 values: 4 from the quaternion (qx,qy,qz,qw), 3 accel, 3 gryo. The quaternion data is from the DMP and type is float, while the accel and gyro are of type int. This is what the code looks like for sending the imu data:

case IMU_READ:

    Serial.print(q.w);
    Serial.print(" ");
    Serial.print(q.x);
    Serial.print(" ");
    Serial.print(q.y);
    Serial.print(" ");
    Serial.print(q.z);
    Serial.print(" ");
    Serial.print(ax);
    Serial.print(" ");
    Serial.print(ay);
    Serial.print(" ");
    Serial.print(az);
    Serial.print(" ");
    Serial.print(gx);
    Serial.print(" ");
    Serial.print(gy);
    Serial.print(" ");
    Serial.println(gz);
  
    break;

The entire code for the arduino is here: ros_arduino_bridge ros_arduino_bridge works absolutely fine, I've just configured it to use the IMU DMP through the MPU6050 library. Everything else related to the DMP works fine except the sending of the data over serial.

From the ROS side, I have to parse the data and store it. I am doing it with this function here.

void read_imu(double& qx, double& qy, double& qz, double& qw, int& ax, int& ay, int& az, int& gx, int& gy, int& gz)
  {
    std::string response = send_msg("i\r"); // returns the imu string data from the arduino
    std::string delimiter = " ";
    size_t start_pos = 0;

    for (int i = 0; i < 9; ++i) {
        size_t del_pos = response.find(delimiter, start_pos);

        // Assign each value directly to its respective variable
        switch (i) {
            case 0: 
              try {
                qx = std::stod(response.substr(start_pos, del_pos - start_pos).c_str());;
              } 
              catch (const std::invalid_argument&) 
              {
                std::cerr << "Argument is invalid\n";
                throw;
              } 
              break;
            case 1: qy = std::stod(response.substr(start_pos, del_pos - start_pos).c_str()); break;
            case 2: qz = std::stod(response.substr(start_pos, del_pos - start_pos).c_str()); break;
            case 3: qw = std::stod(response.substr(start_pos, del_pos - start_pos).c_str()); break;
            case 4: ax = std::stoi(response.substr(start_pos, del_pos - start_pos).c_str()); break;
            case 5: ay = std::stoi(response.substr(start_pos, del_pos - start_pos).c_str()); break;
            case 6: az = std::stoi(response.substr(start_pos, del_pos - start_pos).c_str()); break;
            case 7: gx = std::stoi(response.substr(start_pos, del_pos - start_pos).c_str()); break;
            case 8: gy = std::stoi(response.substr(start_pos, del_pos - start_pos).c_str()); break;
        }

        start_pos = del_pos + delimiter.length();
    }

    // Extract the last value
    gz = std::atoi(response.substr(start_pos).c_str());
}

The problem is that the whole thing is very slow and either the Serial read times out or there is some data missing. I'm at a loss for how to speed it up or send data efficiently as I'm still new to this so any guidance is helpful. I'm using an Arduino Mega with a baud rate of 115200 (p.s I know ros_arduino_bridge is written for the UNO and not Mega, but the IMU bit I'm doing is independent of the ports and I will port it for the Mega once I get this working).

I'm sorry but I'm a bit confused. Could you please elaborate?

show what is printed

I don't have access to the arduino right now but this is pretty similar to what gets printed.

1.00 0.00 0.18 0.56 2456 134 -35 13 52 40

ok, but "void read_imu()" is on receiver side right?

Yes, it is on the computer and is reading serial data from the arduino through the usb port

computer, i see, type double is no problem then.
receiver program in C or C++?

Yes, C++

this string contain entire data with ending "\r\n\r", right?

Yes

@jremington sorry for pinging you, but if you have any idea advice or tips it would be greatly appreciated.

Serial data transmission can be fast and easy with a bit of advanced planning, like having easily recognizable delimiters for the start and end of each packet.

See the Serial Input Basics tutorial for some ideas.

The problem is that the whole thing is very slow and either the Serial read times out

The receiver should handle the input stream on a character-by-character basis using Serial.available(), and with appropriate logic this won't happen.

1 Like

115 k means about 12 characters/ms; so it will take about 4 ms to send your data.
How frequently are you sending this - is it in a loop as fast as possible, or is it a timed cycle? The outbound serial buffer, depending on which Arduino, is likely 64 characters long. If you push characters into it faster than it empties, you'll end up rate-limited, as the Arduino waits for transmissions.

Your code is extremely "clunky"(forgive me), with calling print for every space, etc. Look into the concept of printing into a buffer, then using print to send the buffer.
For debugging, try tracking loop execution times using a millis() calculation, and substitute the loop execution time for one of your variables.

Also, are any/all of your numerics precision-limited? You seem to be only displaying two decimal places, is that by intent?

Are your ints two or four byte? Which Arduino is doing the sending?

Finally, consider sending binary data.
Questions, questions.

I don't mind you insulting the code lol, in fact I encourage you to do so because I can find out where I went wrong.

I don't have any control over the speed of the loop, basically ROS reads the data in as fast as possible because the whole point is for it to be in real time with minimal delay.

The numbers aren't precision limited (but preferably, it should be 2 decimal places since I don't need a lot of accuracy). The reason that I only displayed 2 decimal places is because, from what I've seen, the DMP only outputs up to decimal places.

Could you please provide links for the 'outbound' buffer. I didn't know that the Arduino had one, as far as I'm aware, I know that the Arduino has buffer for receiving data.

The arduino is a mega, and I'm unaware of the size of the int. I'm just using the default int variables that you get when you simply type 'int'.

Could you please explain the binary data part, I'm not sure how well binary data will work because I also have to send decimals, and it would make the number of bytes transferred over serial longer. I also have to quickly convert it on the receiving end so transferring binary might be slower, maybe? idk.

Thank you for the link, I will study it. Could you please elaborate on "having easily recognizable delimiters for the start and end of each packet"? The delimiter I'm using is just a simple " ".

Read the tutorial.

" " is NOT an easily recognizable start or end of packet, as it is everywhere in your data.

Thank you for your help again :slight_smile: !

#include <iostream>
using namespace std;

int main()
{
    string response ="1.00 0.00 0.18 0.56 2456 134 -35 13 52 40\r\n"; 
    string delimiter = " ";
    size_t start_pos = 0;

    for (int i = 0; i < 9; ++i) {
        size_t del_pos = response.find(delimiter, start_pos);
        if (i<4)cout << stof(response.substr(start_pos, del_pos - start_pos).c_str())<<endl;
        else cout << stoi(response.substr(start_pos, del_pos - start_pos).c_str())<<endl; 

        start_pos = del_pos + 1;
    }
    cout << atoi(response.substr(start_pos).c_str())<<endl;
    return 0;
}
1
0
0.18
0.56
2456
134
-35
13
52
40

lets say that the binary equivalent od 12345.67 is 0x12345678, you would still have to serial.print it right? So it would take 10 bytes to send this data through right? Or do you mean to print out the ascii value of each byte i.e 0x12 = A, 0x34 = B etc.. so you print out A B C D (or ABCD to save the 4 bytes in the middle due to the space)?

in that case what would you do when the byte value is greater than 127? i.e 213? then how would you map it to ascii?