Use both software- and hardware serial - slow and/or no output.

I have a project (my first Arduino project) where I’m making a GPS tracker. The intended features I have not added yet, I’m at the point where I’m struggling to get both the GPS- and GPRS unit to work simultaneously and output some meaningful data.

I have one SIM900 GSM module, and one NEO-6M GPS unit, in addition to the UNO board. If I separate the code for the GPS- and GSM part, they individually work very well. The GPS outputs coordinates to the Serial monitor, and the GSM part (so far) sends an SMS.

The problem then seems to be combining the two, the GPS unit stops sending the coordinates to the serial monitor. Specifically, it shows them every once in a while if I wait long enough, maybe one every minute or so. I just cannot seem to figure out why this behaviour occurs, could it be some interrupt thing using both serial channels?

I understand that the UNO cannot have two software serial lines, and as such, I’m using the hardware serial for the GPS unit (which works fine if I exclude the GSM code) (D0 and D1). The GSM unit uses the software serial pin 7 and 8. I’m also using the NeoGPS library for GPS communication, as the TinyGPS library could not decode the information received correctly.

In the code below, I’m starting the Arduino, send one SMS, and after that the GPS is supposed to output the coordinates continuously to the serial monitor. The serial monitor continuously outputs any new data from GSM, i.e. new SMS received.

Full code:

#include <NMEAGPS.h>
#include <GPSport.h>
#include <SoftwareSerial.h>

NMEAGPS  gps; // This parses the GPS characters
gps_fix  fix; // This holds on to the latest values

//Create software serial object to communicate with SIM900
SoftwareSerial mySerial(7, 8); //SIM900 Tx & Rx is connected to Arduino #7 & #8

void setup()
{
  Serial.begin(9600);
  gpsPort.begin(9600);
  mySerial.begin(9600); //Begin serial communication with SIM900

  Serial.println("Initializing GSM..."); 
  delay(5000);

  mySerial.println("AT"); // Handshaking with SIM900
  updateSerial();
  mySerial.println("AT+CMGF=1"); // Configuring TEXT mode
  updateSerial();
  mySerial.println("AT+CNMI=1,2,0,0,0"); // Decides how newly arrived SMS messages should be handled
  updateSerial();

  sendSMS();

}

void loop()
{  
  
  while (gps.available( gpsPort )) {
    fix = gps.read();

    if (fix.valid.location) {
      
      Serial.print("Location: ");
      Serial.print( fix.latitude(), 6 );
      Serial.print( ',' );
      Serial.print( fix.longitude(), 6 );
      Serial.println();

    }else{
      Serial.print("No GPS fix. \r\n");
    } 
  }

  updateSerial();
}


  void updateSerial()
{
  delay(500);
  
  while(mySerial.available()) 
  {
    Serial.write(mySerial.read()); //Forward what SIM900 received to Serial Port
  }
}

void sendSMS()
{
 
  mySerial.println("AT+CMGS=\"+<removed>\"");
  updateSerial();
 
  mySerial.print("Shit is not working..");
  updateSerial();
  
  mySerial.write(26);
  updateSerial();
 
}

The other part I don’t quite understand is the following. It seems that I have to flush the incoming serial buffer for every “AT command” sent to the GSM unit, otherwise nothing happens. It slo works just putting “mySerial.read()” in the code not assigning it to anything. What is the reason for flushing the incoming serial buffer in this case? If I remove the “Serial.write” line below no SMS is sent.

  while(mySerial.available()) 
  {
    Serial.write(mySerial.read()); //Forward what SIM900 received to Serial Port
  }

I’m also aware that I cannot load the Arduino with hardware serial connected, so I disconnect the GPS unit every time I load modifications to the program.

Would I be better off using the Arduino Mega instead to have the extra serial ports?

tormods:
Would I be better off using the Arduino Mega instead to have the extra serial ports?

Yes!!!

Power_Broker:
Yes!!!

That can be said without even looking at the code. Doesn't have to be a Mega though. There are many board in the Arduino Ecosystem that have multiple hardware serial ports.

1284P has 2 hardware serial ports. 1280, 2561 and 2560 have 4.

Ok, it seems I’ll be ordering a second board then :slight_smile:

Having said that, I’m building this project to learn, so I would still be interested in knowing why this code doesn’t work properly. Especially that part with flushing the incoming buffer which I don’t understand why needs to be there.

If the GPS is connected to pins 0 and 1, every time that you do a serial print, you also send it to the GPS.

On the Uno, pins 0 and 1 carry the same signals as used for the communication with the PC.

sterretje:
If the GPS is connected to pins 0 and 1, every time that you do a serial print, you also send it to the GPS.

On the Uno, pins 0 and 1 carry the same signals as used for the communication with the PC.

Yes, that is my setup, GPS connected to pin 0 and 1. It makes sense though, as you say, the GPS getting clogged by useless commands. I though maybe unknown commands would be discarded. So it would only work then, if nothing prints to pin 0 and 1, except commands to and from the GPS module?

Anyways, I put in an order for a MEGA board, so I'll try again then (only drawback is the bigger size of that board, was hoping to make the package smaller).

What are other options for GPS receivers, could I use something else than serial communication if I got another GPS unit?

tormods:
Anyways, I put in an order for a MEGA board, so I’ll try again then (only drawback is the bigger size of that board, was hoping to make the package smaller).

As I noted in Reply #2, there are many other choices besides a Mega – it depends mostly on what your other I/O requirements are. If you want to stick with AVR architecture, consider a 32u40-based board. Like ATMega328P, it also only has one hardware serial port. But you’re free to use it for anything you want (as Serial1) because connection to computer for code download and serial debugging (aka Serial) is provided by a native USB port. Adafruit as several small, low cost variants:

If you’re interested in moving up to a 32-bit ARM architecture, there are a large number of choices.

gfvalvo:
As I noted in Reply #2, there are many other choices besides a Mega -- it depends mostly on what your other I/O requirements are. If you want to stick with AVR architecture, consider a 32u40-based board. Like ATMega328P, it also only has one hardware serial port. But you're free to use it for anything you want (as Serial1) because connection to computer for code download and serial debugging (aka Serial) is provided by a native USB port. Adafruit as several small, low cost variants:
Overview | Introducing ItsyBitsy 32u4 | Adafruit Learning System
Overview | Adafruit Feather 32u4 Basic Proto | Adafruit Learning System

If you're interested in moving up to a 32-bit ARM architecture, there are a large number of choices.

Hi gfvalvo, and thanks for your reply. The adafruit ones looks indeed interesting, looks a lot like one of those Arduino Nanos? What do you mean by "native USB port", is the PC-board communication separate, so I can use the Hardware serial output for communication only, and other USB serial line for monitoring (serial monitor output) and application download? So more of a dedicated hardware serial input/output?

I would like to stick with the AVR architecture yes. As I said, I'm quite new to this, and although I have a software background I know pretty much nothing when it comes to circuits and electronics, so I'd like to stick with Arduino compatible devices, makes it easier to use examples and find information and help on the internet :slight_smile: I'm doing my best, but don't want to move too quickly ahead.. trying to learn what I can :slight_smile:

Btw, The Arduino UNO I'm using now has an ATMega328P. Does this mean it has a dedicated hardware serial as you mentioned already?

tormods:
What do you mean by "native USB port"...

Just that. The 32u4 chip has a built in USB port that can appear on to your PC as a USB Serial COM Port. This is how you download code and do debugging (i.e. Serial.print()). It does this without the benefit of a helper processor, SiLabs chip, or FDDI chip to convert from hardware USART to USB. It can also be made to appear to your PC as a HID.

That leaves the hardware USART for connection to your GPS. It it will be Serial1 in your code.

Btw, The Arduino UNO I'm using now has an ATMega328P. Does this mean it has a dedicated hardware serial as you mentioned already?

Yes, but it's dedicated to code downloading and Serial.print() via a helper processor that converts from USART serial to USB Serial. That makes it unavailable to connect to your GPS.

gfvalvo:
Just that. The 32u4 chip has a built in USB port that can appear on to your PC as a USB Serial COM Port. This is how you download code and do debugging (i.e. Serial.print()). It does this without the benefit of a helper processor, SiLabs chip, or FDDI chip to convert from hardware USART to USB. It can also be made to appear to your PC as a HID.

That leaves the hardware USART for connection to your GPS. It it will be Serial1 in your code.

Yes, but it's dedicated to code downloading and Serial.print() via a helper processor that converts from USART serial to USB Serial. That makes it unavailable to connect to your GPS.

Aha, so that means that I can actually use the Arduino Leonardo or the Micro? It seems they have all I need, two serial communication lines (one software and one hardware that hopefully works) and a few more pins for some LEDs. I'm also asking because my local dealer has both in stock, I would be tempted to get one to try this right away :slight_smile:

Anyway, thanks for the helpful insight :slight_smile:

If you you look on their product pages, you'll see that both Leonardo and Micro are based on the 32u4 chip.

gfvalvo:
If you you look on their product pages, you’ll see that both Leonardo and Micro are based on the 32u4 chip.

Ok, so I picked up a Leonardo board, and while my two serial lines now works perfectly (thanks!), I’m struggling to understand how the “mutiple serial communication” works (I think).

The GPS is connected to Serial1, and the following (very simple) code works perfectly:

void setup(){
  Serial.begin(9600); //Serial monitor
  Serial1.begin(9600); //GPS receiver
}

void loop(){
  while (Serial1.available() ){
    byte gpsData = Serial1.read();
    Serial.write(gpsData);
  }
}

This gives me the output from the GPS sensor as follows (I’ve replaced my actual coordinates). So it receives the data, and TinyGPS++ library is also able to parse this data, so the format is OK:

$GPVTG,,T,,M,0.392,N,0.727,K,A*29
$GPGGA,193411.00,6721.92245,N,00720.20407,E,1,03,2.59,87.0,M,43.8,M,,*65
$GPGSA,A,2,26,05,29,,,,,,,,,,2.77,2.59,1.00*04
$GPGSV,3,1,12,04,05,290,,05,22,040,27,09,11,320,,16,51,286,23*70
$GPGSV,3,2,12,20,04,151,,21,59,159,,23,05,296,,25,02,133,*77
$GPGSV,3,3,12,26,70,239,20,27,16,258,,29,40,083,28,31,20,206,*74
$GPGLL,6721.92245,N,00720.20407,E,193411.00,A,A*6D
$GPRMC,193412.00,A,6721.92252,N,00720.20350,E,0.204,,120220,,,A*71
$GPVTG,,T,,M,0.204,N,0.377,K,A*26
$GPGGA,193412.00,6721.92252,N,00720.20350,E,1,03,2.59,87.0,M,43.8,M,,*65
$GPGSA,A,2,26,05,29,,,,,,,,,,2.77,2.59,1.00*04
$GPGSV,3,1,12,04,05,290,,05,22,040,27,09,11,320,,16,51,286,22*71
$GPGSV,3,2,12,20,04,151,,21,59,159,,23,05,296,,25,02,133,*77
$GPGSV,3,3,12,26,70,239,20,27,16,258,,29,40,083,28,31,20,206,*74
$GPGLL,6721.92252,N,00720.20350,E,193412.00,A,A*6D
$GPRMC,193413.00,A,6721.92273,N,00720.20321,E,0.215,,120220,,,A*75
$GPVTG,,T,,M,0.215,N,0.398,K,A*27
$GPGGA,193413.00,6721.92273,N,00720.20321,E,1,03,2.58,87.0,M,43.8,M,,*67

Now, if I add a small delay to the code, the entire output is messed up, and the library is now not able to parse the data anymore:

void setup(){
  Serial.begin(9670); //Serial monitor
  Serial1.begin(9670); //GPS receiver
}

void loop(){
  while (Serial1.available() ){
    byte gpsData = Serial1.read();
    Serial.write(gpsData);
  }

  delay(500);
}

Why would a small delay ruin the output? It’s not a part of the “read loop” as long as data is available in the serial receive buffer? I have read the tutorials page regarding serial communication, and from what I can gather, one answer might be that, the “while loop” actually empties the buffer quicker than what is transmitted from the GPS sensor, and hence it will exit the loop and run the remaining code. If so I still don’t understand why the output is all of a sudden disorganised with extra line-breaks and whatnot.

The output now looks like this:

$GPRMC,193509.00,A,6721.92195,N,00720.20407,E,0.402,,120220,,,A*6B
$GPRMC,193510.00,A,6721.92145,N,00720.20531,E,1.543,,120220,,,A$GPRMC,193511.00,A,6721.92170,N,00720.20628,E,1.879,,120220,,,A
$GPRMC,193512.00,A,6721.92189,N,00720.20592,E,0.538,,120220,,,A$GPRMC,193513.00,A,6721.92108,N,00720.20695,E,1.292,,120220,,,A$GPRMC,193514.00,A,6721.92104,N,00720.20687,E,0.570,,120220,,,A
$GPRMC,193515.00,A,6721.92072,N,00720.20752,E,1.207,,120220,,,A$GPRMC,193516.00,A,6721.92069,N,00720.20936,E,1.465,,120220,,,A$GPRMC,193517.00,A,6721.92016,N,00720.20997,E,1.406,,120220,,,A$GPRMC,193518.00,V,,,,,,,120220,,,N*79
$GPVTG,,,,,,,,,N*30
$GPGGA,193$GPRMC,193519.00,V,,,,,,,120220,,,N*78
$GPVTG,,,,,,,,,N*30
$GPGGA,1935$GPRMC,193520.00,V,,,,,,,120220,,,N*72
$GPVTG,,,,,,,,,N*30
$GPGGA,193520$GPRMC,193521.00,V,,,,,,,120220,,,N*73
$GPVTG,,,,,,,,,N*30
$GPGGA,193521.0$GPRMC,193522.00,V,,,,,,,120220,,,N*70
$GPVTG,,,,,,,,,N*30
$GPGGA,193522$GPRMC,193523.00,V,,,,,,,120220,,,N*71
$GPVTG,,,,,,,,,N*30

I started to play around with some delays due to an SMS function I was to add, which brings me to the next part I don’t understand. If I add the code for receiving SMS, the received SMS appears in between all the other data from the GPS sensor:

(GSM unit connected to pin 8,9)

#include <SoftwareSerial.h>

SoftwareSerial mySerial(8, 9); //SIM900 Tx & Rx is connected to Arduino #7 & #8

void setup(){
  Serial.begin(9600); //Serial monitor
  Serial1.begin(9600); //GPS receiver

  mySerial.begin(9600);

  delay(2000);

  Serial.println("Initializing GSM..."); 
  delay(5000);

  mySerial.println("AT"); // Handshaking with SIM900
  updateSerial();
  mySerial.println("AT+CMGF=1"); // Configuring TEXT mode
  updateSerial();
  mySerial.println("AT+CNMI=1,2,0,0,0"); // Decides how newly arrived SMS messages should be handled
  updateSerial();
}

void loop(){
  while (Serial1.available() ){
    byte gpsData = Serial1.read();
    Serial.write(gpsData);
  }
  updateSerial();
}

  void updateSerial()
{
  while(mySerial.available()) 
  {
    Serial.write(mySerial.read()); //Forward what SIM900 received to Serial Port
  }
}

And the result:

$GPGSV,3,1,12,05,06,019,18,07,16,333,,08,12,273,,10,08,158,*71
$GPGSV,3,2,12,13,04,046,,15,02,074,,16,68,232,16,20,30,139,17*7B
$GPGSV,3,3,12,21,65,095,23,26,51,184,12,27,45,268,,29,12,093,20*71
$GPGLL,6321.92679,N,00720.24175,E,204059.00,A,A*67
$GPRMC,204100.00,A,6321.92677,N,00720.24174,E,0.559,,120220,,,A*76
$GPVTG,,T,,M,0.559,N,1.035,K,A*2D
$GPGGA,20
4
+1C00M.T0:0 ,"6+047219.39628667576,4N",,0"0"5,"202.02/4012/741,2E,,211:,0440,:35.62+10,4"95
.
[b]4A,RMd,u4i3no.8 ,fMo,r,um*6 3h
e
e$eGlPpG[/b]
SA,A,3,20,21,05,29,,,,,,,,,4.00,3.21,2.38*00
$GPGSV,3,1,12,05,06,019,18,07,16,333,,08,12,273,,10,08,158,*71
$GPGSV,3,2,12,13,04,046,,15,02,074,,16,68,232,14,20,30,139,17*79
$GPGSV,3,3,12,21,65,095,23,26,51,184,12,27,45,268,,29,12,093,21*70
$GPGLL,6321.92677,N,00720.24174,E,204100.00,A,A*65
$GPRMC,204101.00,A,6321.92677,N,00720.24086,E,0.435,,120220,,,A*70

Highlighted part is the received SMS “Arduino forum heelp” :slight_smile:

Do the Arduino only have one common serial buffer, which is flooded by whatever is transferring the data (independent of the source)? And if so, how do I separate the two?

I want to receive proper GPS data and parse it, additionally I would like to act upon SMS messages received…

Each hardware serial port has its own 64-byte ring buffer. A common rule is that any sketch that deals with serial communication must be fast/efficient enough to handle data at least as fast as it arrives to avoid buffer overflow. When the ring buffer is full, newly received bytes are dropped and that's what seems to be happening in your sketches when using delay() (which is bad to use anyway).

Power_Broker:
Each hardware serial port has its own 64-byte ring buffer. A common rule is that any sketch that deals with serial communication must be fast/efficient enough to handle data at least as fast as it arrives to avoid buffer overflow. When the ring buffer is full, newly received bytes are dropped and that's what seems to be happening in your sketches when using delay() (which is bad to use anyway).

Aha, that makes sense I guess. So the buffer fills up in the background while remaining code halts for the specified amount of time?

Do both software- and hardware serial have their own buffer then? Since I'm using one sw- and one hardware?

Also, I'm unsure how that explains why both buffers are mixed up in the print? Is this because, the "buffer read()" code empties the queue faster then its filled up, and skips between the two read() routines?

Does this mean that if I, as opposed to print the result for verification, assign it to a variable and use it for further processing all data would then be intact? Meaning, Its just the print that making me confused..?

Sorry for the newbie questions. I'm really trying to get my head around this. I really appreciate the help!

Power_Broker:
...

gfvalvo:
...

Thanks guys for all the help you have provided me. I've completed the rest of the project on my own now, and since I appreciate an active and helpful community, I've tried to give something back by publishing my finished project at project hub: https://create.arduino.cc/projecthub/tormods/car-gps-tracker-with-thinger-io-iot-integration-b996b3

Take care :slight_smile: