Arduino Mega Bluetooth Commands Slow

I've been working on this problem for a couple weeks, and still stumped!

My project is to add Bluetooth capability to an existing project which is an Arduino MEGA that controls the band switching on my Amplifier. In the past, this has been through the Serial Ports on the MEGA. My new radio has Bluetooth capabilities for reading the frequency.

The problem that I'm having is that the response from the Radio is Slow, often 4-5 seconds for a response from the command over Bluetooth. I was using a NANO/HC-05 with SoftwareSerial.h, when it occurred to me this is SLOW! So I replaced the NANO and used a spare MEGA via Serial3, native to the MEGA, but the commands are still Slow and erratic. In this case, the MEGA is ONLY running my test code.

I ran basically the same code via an ESP32 WROVER and the commands respond NOW!!! The trouble is, I need to use the HC-05 with the MEGA in my project.

  1. Can someone point me in the right direction as to how to get a faster response, or what I may be doing wrong? i.e.: Why is the ESP32 so responsive?

  2. Is there a way to tell what baud rate the Bluetooth is communicating at? I'm using the setup() statement: Serial3.begin(38400); to write to the HC-05 (TX has resistor divider).

  3. Do I need to change the default baud rate in the HC-05? When I set it to "AT+UART=38400,0,0" it responds with "38400,,,,,". And after reboot, it's defaulted back to "4800,0,0"

Sir Michael

Please show the sketch.

Sorry, I guess I should have posted my code, but it seems fairly involved...

As I mentioned, when I load the same file (modified slightly) into the ESP32, the Read Frequency command is answered quite promptly. When I read through the MEGA(2560) & the HC-05, I (typically) get a 4 second timeout, retry, another command retry, and a 4 second timeout, and finally the response.

Because of the delays, I had to build in the "WAIT" period to wait for the response. (kludge...)

When I spin the dial (the radio transmits each update, quite quickly). The ESP32 seems to be able to keep up, whereas the MEGA/HC-05 drops out after a few seconds and picks up again...


void setup()
{
  Serial.begin(115200);
  Serial3.begin(38400);  //Default Baud for BT comm
  Serial.println("Setup");
  Serial.println(F(" 'r' Read Frequency,"));
  Serial.println(F(" 'p' processCatMessages."));
}

//IC-705 Settings:
// Settings, Bluetooth, Data Device Set, Serialport Function: CI-V(Echo Back OFF)

void loop() {
  ReadMonitor();
}


void ReadMonitor() {

  //Read the Character from the input:
  if (Serial.available() > 0) {    // is a character available?
    unsigned long StartTime = millis();
    uint32_t rx_byte = Serial.read();       // get the character

    // check if a Char was received
    if (rx_byte == 'r') {
      Serial.println(F("COMMAND: 'r', Read The Frequency."));
      uint32_t Freq = CIV_Read_Frequency();
      Serial.print(F("OUTCOME: Frequency = ")); Serial.println(Freq);
      Serial.println();
    }
    else if (rx_byte == 'p') {
      Serial.println(F("COMMAND: 'p', ProcessCatMessages (60 seconds)."));
      do {
        uint32_t Frequency = processCatMessages(false);
        if (Frequency > 1000) {
          Serial.print(F(" Freq = ")); Serial.println(Frequency);
        }
        //When the CheckSpin is done, re-read the frequency:
        if (CheckSpin()) {
          Serial.print(F(" CheckSpin Timeout: Freq = ")); Serial.println(Frequency);
          //Manually read the frequency:
          Frequency = CIV_Read_Frequency();
        }
        delay(10);
      } while ((millis() - StartTime) < 60000);
      Serial.println(F("OUTCOME: 60 Seconds Complete "));
      Serial.println();
    }
    Serial.print(F("    Runtime = ")); Serial.println(millis() - StartTime);  Serial.println();
  } // end: if (Serial.available() > 0)
}

The Other file with all the meat in it...


uint8_t  RADIO_ADDRESS = 0xA4;     //Transiever address
const byte BROADCAST_ADDRESS = 0x00; //Broadcast address
const byte CONTROLLER_ADDRESS = 0xE0; //Controller address
const byte START_BYTE = 0xFE; //Start byte
const byte STOP_BYTE = 0xFD; //Stop byte
const byte CMD_READ_FREQ = 0x03; //Read operating frequency data
const byte CMD_TRANSCEIVE_FREQ = 0x00;
const byte AckMsg = 0xFB; //Received an ACK from the ICOM rig.
const byte NacMsg = 0xFA; //Receive a "Not Good" message from the rig.
const uint32_t decMulti[] = {1000000000, 100000000, 10000000, 1000000, 100000, 10000, 1000, 100, 10, 1};
uint32_t  readtimeout = 5000;      //Serial port read timeout, Not sure what to set at, trying 5 seconds. (Mike 10/23/21)
int gDialSpin = 0;  // Transceive mode, we stop decoding the frequency, then read Frequency 5 seconds later after the updates stop.

uint32_t processCatMessages(boolean WaitCmd) {
  //Set WaitCmd to 'true' if a SPECIFIC command was sent, and you are waiting for a response.
  //Set WaitCmd to 'false' if you are just checking the buffer (i.e.:Transceive Mode).
  //Returns the data Read from: Frequency,

  uint8_t read_buffer[12];   //Read buffer
  byte Count = 0;

  //A SPECIFIC command was sent, wait for the response!
  if (WaitCmd) {
    unsigned long StartWait = millis();
    do {
      delay(20);
      Count++;
    } while ((Serial3.available() == 0) && (Count < 200)); //Count limited to 255

    Serial.print(F(" processCatMessages Count = ")); Serial.print(Count); Serial.print(F(" Out of 200."));
    Serial.print(F("   Wait Time = ")); Serial.println(millis() - StartWait);
    if (Serial3.available() == 0) return 0;
  }

  while (Serial3.available()) {
    if (readLine(read_buffer, sizeof(read_buffer)) > 0) {
      if (read_buffer[0] == START_BYTE && read_buffer[1] == START_BYTE) {
        if ((read_buffer[3] == RADIO_ADDRESS) || (read_buffer[4] == RADIO_ADDRESS)) {
          if ((read_buffer[2] == BROADCAST_ADDRESS) || (read_buffer[2] == CONTROLLER_ADDRESS)) {
            switch (read_buffer[4]) {
              case CMD_TRANSCEIVE_FREQ: //0x00
              case CMD_READ_FREQ: { //0x03
                  if (gDialSpin == 0) {
                    //Dial is not spinning, decode the frequency:
                    Serial.print(F("gDialSpin =")); Serial.print(gDialSpin);
                    gDialSpin++;
                    Serial.print(F(" gDialSpin updated to: =")); Serial.println(gDialSpin);
                    return printFrequency(read_buffer);
                  }
                  else {
                    //Add to the global int gDialSpin.
                    gDialSpin++;
                    Serial.print("*");
                    return 0;
                  }
                  break;
                }
              case AckMsg: { //Ack Message (FB) from ICOM
                  Serial.println(F("  FB Recieved, Returned 255."));
                  return 255;
                }
              case NacMsg: { //Ack Message (FA) from ICOM
                  Serial.println(F("  FB Recieved, Returned 254."));
                  return 254;
                }
              default: {
                  //Any other messages that may come through, print them out...
                  Serial.print("           ##  ERROR?       Default <");
                  for (uint8_t i = 0; i < sizeof(read_buffer); i++) {
                    if (read_buffer[i] < 16)Serial.print("0");
                    Serial.print(read_buffer[i], HEX);
                    Serial.print(" ");
                    if (read_buffer[i] == STOP_BYTE) break;
                  }
                  Serial.println(">");
                  break;
                }
            }
          }
        }
      }
    }
  }
  return 0;
}



uint8_t readLine(uint8_t buffer[], uint8_t ArraySize) {
  //This function reads the line from the CIV Buffer and returns it in the 'buffer[]' array.
  //  NOTE: I don't think the Timeout is required at all.
  uint8_t NextByte;
  uint8_t counter = 0;
  uint32_t ed = readtimeout;

  while (true)
  {
    while (!Serial3.available()) {
      if (--ed == 0)return 0;
    }
    ed = readtimeout;
    NextByte = Serial3.read(); // Read 1 byte at a time.
    // NOT SURE WHAT THIS DOES, I don't think the '0xFF' is used! Mike 11/6/21
    //if (NextByte == 0xFF) continue; //TODO skip to start byte instead
    buffer[counter++] = NextByte;
    if (STOP_BYTE == NextByte) break;
    if (counter >= ArraySize) return 0;
  }

  //Temporarily, print out ALL lines that are received.
  Serial.print("               ####### readLine Data<");
  for (uint8_t i = 0; i < counter; i++) {
    if (buffer[i] < 16)Serial.print("0");
    Serial.print(buffer[i], HEX);
    Serial.print(" ");
    if (buffer[i] == STOP_BYTE)break;
  }
  Serial.println(">");

  return counter;
}


uint32_t printFrequency(uint8_t buffer[]) {
  uint32_t frequency = 0;
  //Freq<FE FE E0 A4 03 00 00 00 01 00 FD>  1.0 MHz
  //Freq<FE FE E0 A4 03 00 00 00 33 04 FD>  433.0 MHz
  //Start at the Right most digit, read_buffer[9]  (MSD):
  for (uint8_t i = 0; i < 5; i++) {
    if (buffer[9 - i] == 0xFD) continue; //End of Message, break out of loop...
    frequency += (buffer[9 - i] >> 4) * decMulti[i * 2];
    frequency += (buffer[9 - i] & 0x0F) * decMulti[i * 2 + 1];
  }
  //We divide by 1000 to give freq in KHz. Precision is not requried.
  frequency = frequency / 1000;
  return frequency;
}

void CivSendCmd(uint8_t buffer[], uint8_t ArraySize) {
  //This functiobn is used to SEND the commands (from the 'buffer[]' to the CIV port
  Serial.print(F("  SEND CMD: <"));
  for (uint8_t i = 0; i < ArraySize; i++) {
    Serial3.write(buffer[i]);
    Serial.print(buffer[i], HEX); Serial.print(" ");
  }
  Serial.println(F("> SEND Command Complete"));
}

boolean WaitOkMsg(void) {
  //If a command was send that does NOT expect specific data return, then we wait for the OK Message (FB) return data.
  //  If the command failed, then the Message NG "FA" will be returned.
  uint8_t Return;

  Return = processCatMessages(true);

  if (Return == 255) {
    Serial.println(F(" GOT RETURN OF 255 After Command"));
    return false;
  }
  else if (Return == 254) {
    Serial.println(F(" ERROR, NG CODE OF 254 (NacMsg) After Command"));
    return true;
  }
  else {
    Serial.println(F(" ERROR, NO RETURN of OK Message After Command"));
    return true;
  }
}

boolean CheckSpin(void) {
  //When the dial is spinning, the frequencies change so fast, we can't keep up (don't need to!), so we stop decoding the Frequencies.
  //  This function is only called when the Frequency is changed in Transceive mode.

  static unsigned long StartSpin;
  static int LastDialSpin;

  if (gDialSpin == 0) {
    //Frequency changed and global gDialSpin==0
    //  Start the timer
    StartSpin = millis();
    LastDialSpin = gDialSpin;
    return false;
  }
  else {
    //Frequency changed again, or we are just checking for state...
    if (((millis() - StartSpin) > 5000) && (LastDialSpin == gDialSpin)) {
      //Return true will trigger a manual Read of the Frequency.
      Serial.println(F("   $$$$ Return Timeout, > 5 Sec."));
      gDialSpin = 0;
      StartSpin = millis();
      return true;
    }
    if (LastDialSpin != gDialSpin) {
      Serial.print(F("  LastDialSpin="));
      Serial.print(LastDialSpin);
      Serial.print("/");
      Serial.println(gDialSpin);
    }
    LastDialSpin = gDialSpin;

    return false;
  }
}


uint32_t CIV_Read_Frequency(void) {
  //Don't Read the message, it will be picked up by the normal processCatMessages() loop...

  //Construct the message to read the frequency
  byte ReadFreq[] = {START_BYTE, START_BYTE, RADIO_ADDRESS, CONTROLLER_ADDRESS, CMD_READ_FREQ, STOP_BYTE};
  uint32_t ReturnFreq = 0;
  byte Count = 0;
  do
  {
    //Send the message
    CivSendCmd(ReadFreq, sizeof(ReadFreq));
    //Decode the returned message
    ReturnFreq = processCatMessages(true);
    Serial.print(F(" CIV_Read_Frequency read: ")); Serial.print(ReturnFreq); Serial.print(F(" Count = ")); Serial.println(Count);
    Count++;
  } while ((ReturnFreq < 1000) && (Count < 5));  //Expect the Frequency to be > 1.0 MHz.
  Serial.print(F(" CIV_Read_Frequency read: ")); Serial.println(ReturnFreq);

  //Manual Read the the frequency always clears the gDialSpin variable.
  gDialSpin = 0;
  return   ReturnFreq;
}

I'm very sorry, but the code is confusing for me. I can not see the overall structure of the code.

There are delay() and while-loops. Sometimes a combination of those. You should avoid this:

      do {
        ...
        delay(10);
      } while ((millis() - StartTime) < 60000);

The Arduino Mega has a 8-bit microcontroller and the compiler defaults to 16-bit signed integer for the numbers. The "60000" in the code above is accepted, but you can make it 60000UL to tell the compiler that it is a unsigned long.

There is even code that does not make sense, for example this:

while (!Serial3.available()) {
  if (--ed == 0) return 0;
}
ed = readtimeout;

There is also an large amount outgoing debug text. The 115200 baud is fast, but there might be a full outgoing buffer sometimes.

When I look into a single variable, I'm already confused. For example gDialSpin has no limit for the maximum value. Since it is a integer, it will behave different on a Mega or ESP32.

Another example is this while-loop:

  byte Count = 0;
  do
  {
    //Send the message
    CivSendCmd(ReadFreq, sizeof(ReadFreq));
    //Decode the returned message
    ReturnFreq = processCatMessages(true);
    Serial.print(F(" CIV_Read_Frequency read: ")); 
    Serial.print(ReturnFreq); 
    Serial.print(F(" Count = ")); 
    Serial.println(Count);
    Count++;
  }
  while ((ReturnFreq < 1000) && (Count < 5));    //Expect the Frequency to be > 1.0 MHz.

That is bad coding.

  1. It outputs a lot of text. Also in the functions CivSendCmd() and processCatMessage(). The Serial has a output buffer of 64 bytes. If that buffer is full, the sketch could slow down hundred times.
  2. The processCatMessage(true) is called with true as parameter. That means it could take 4 seconds before that function returns !

What I mention so far, is just by looking at random pieces of code. I see these problems everywhere.

I'm not surprised that it runs in a different way on a ESP32. The combination of baud rate and delay() and while-loops and processor speed can have unforeseen consequences, especially when the overall structure of the code is not clear.

Keopel,

I do appreciate the review of my code.

The delay(10) in the first do {} while loop is so that I can spin the dial, and the code will keep reading the frequency. The 60000 keeps the 'p' (test) function going for 60 seconds, then drops out.

However, in my latest code, I've removed the gDialSpin and all the code that goes with it. It just complicates things, for no really good reason! (Trying to simplify.)

The code that you mentioned that doesn't make sense, was from a previous function that I copied, to get started. I believe it's I agree! I could pull it out I guess, it was apparently for a timeout...

The "byte Count = 0; code will never overflow. I'm only using it to a count of 5 or so...

do {
      delay(10);
      Count++;
    } while ((Serial3.available() == 0) && (Count < 200));

is used to keep trying the while (Serial3.available()==0) until the response is received. I may try to figure out a way to do some other processing until the response is received.

I'm painfully aware of the 64 byte limit of the comms buffer. I fought this in a previous version of this project, and found out how to increase the buffer to 256 bytes. In this case, the buffer is limited to 12 bytes, as declared in the CivSendCmd() and processCatMessage() functions (uint8_t read_buffer[12];).

The thing that still confuses me, is that I'm essentially taking the SAME CODE and putting it into both the ESP32 and the MEGA/HC-05 and I see different results.

On the more positive side, I'm beginning to think that my problem may be in the Baud Rate on the HC-05. I'm currently testing at 57600 and things are improving. From time to time, I'm still seeing some extended delays, and retries.

I have more testing to do...

Thanks,

Sir Michael

I'm not talking about the 12 byte buffer, but the 64 byte buffer inside the Serial library.

It is not hard to fill the 64 byte buffer, even when the baudrate is 115200 baud.
Since it takes about 6 ms to send 64 bytes, the buffer will get full if you try to send more.
Once that buffer is full, the Serial.println() waits until there is a free spot in that buffer. That waiting can slow down a sketch hundred times.

I think that the main problem is the overall structure of the code and the while-loops and the delay().
The sketch would do the same thing on a ESP32, if it was well written and straightforward.

The best way to deal with incoming Serial data is to never wait. Grab incoming data as soon as they are available and put it in a buffer. Process that buffer if a full message is received.
Any waiting for a response can be done with millis().

void loop()
{
  if( Serial3.available() > 0)
  {
    int inChar = Serial3.read();
    buffer[index++] = inChar;

    if( full message is received
    {
      parse and process the message
    }
  }
}

If you use the RTOS of the ESP32, then you can make a task for incoming data, and a task for other things. Then you don't have to use millis(), which is according to some people like wearing a shirt inside out.

Your code is beyond my comprehension, and probably not worth following because Bluetooth is probably innocent anyway, but, for what it's worth, this

is untrue, 38400 is the required rate for HC-05 in AT mode - which you are not doing, and there is no default about it. If you have a speed issue, irrespective of the cause, I submit that you have no good reason not to run Bluetooth at 115200, just as you already are with the serial monitor.

HC-05 can be configured to 1382400 (count the digits), I have never heard of anybody needing to set the second or third parameters.

Koepel,

This project is way too far along to change to an RTOS. (I'm aware of what it is.). It might make some of this easier, but...
I'm trying to add some code to read comms from Bluetooth, and patch it into my existing project.

As far as the buffer's go, I understand that you are talking about the buffers in the Arduino. I have modified HardwareSerial.h to increase the buffer space (for the other devices that comm through serial, there are some commands that are > 64 bytes.

Nick,
I think that you are close to what I've learned today. It does seem to be a difference in the baud rate between the ESP32 and the HC-05. I think I finally have the HC-05 configured for 57600 (and that was a bit tricky, even using a MEGA with real serial ports!

At least at this point, I can get some very reasonable response times for my data queries through Bluetooth.

NOW, what I need to do, is to figure out a way to send a Bluetooth command, let the project MEGA do the rest of it's work, and periodically check for a response. I should be able to have my son (he's a much better programmer than I am) help me figure out a solution.

I thank you both for your guidance!

Sir Michael

I'm betting it's got nothing to do with baud rate at all, and there is something far more fundamental going on. Any Bluetooth facility essentially consists of two transceivers - one to talk via wire to its host computer at the baud rate you set, and the other to talk via wireless to the second device, using Bluetooth protocols that are beyond your control. That second device talks to its host at whatever baud rate you may, or may not be able to set, and which has no bearing on the baud rate on the first device.

I fail to see why you "think" you have HC-05 configured for 57600, it either is or it isn't, and you shouldn't be left in any doubt about either. Further, it should be no more tricky configuring it for that as it is for any other rate.

One problem you might be having is that Bluetooth on the ESP32 is not properly configured for classic Bluetooth SPP mode for comms with HC-05.