Advice on best way to convert NMEA messages

I need to monitor serial input with NMEA messages of various types, forwarding those required as serial output (Arduino pins 1,2). NMEA messages present as a series of variable length comma separated messages, starting with $ and ending with a CR/LF. The last 2 characters are * followed by a checksum consisting of an exclusive or of all characters in the message between the $ and the *, not inclusive of these. The messages occasionally get corrupted, so the length of the incoming string can vary if it is the newline that is lost.

One of the messages, relating to wind speed and angle, is in the new format, MVW, so needs to be converted to its old equivalent, VWR. Therefore the new checksum needs to be calculated, so that it needs to be in an array of char format to do this efficiently.

While I have a long background in computing, starting with assembler in 1971, my only effort with Arduino/C was about 5 years ago when I built an engine monitor to replace the instrument on a boat. This did not involve swapping strings to char and back again, and approaching 79 I am very rusty.

There are some compilation errors in the code below: most of them conflicts between char and string, and the use of the wrong library functions! It is work in progress, as I keep picking it up, and seem to have a number of buffer names. I could persist, but would welcome any input as to the basic strategy to use and chars and strings for the data buffers. I plan to use a Nano as with the engine monitor.

EDITING POST SEVERAL WEEKS LATER

i am posting below, instead of the code that has sat here for some time, the state of play as at October 16, 2024. Thank you for those of you who have taken an interest, particularly J-M-L who is well ahead of me in this game.

I believe that the sketch below is substantially doing what I want. It is validating sentences, filtering out unwanted traffic, and doing the conversion.. We will see if it works in practice, but that is some time away.

//Conversion of MWV to VWR


// October 16 2024      cleaned up debug, appears to be working subject to further testing
// Test messages   $WIMWV,045,R,011.63,N,A*09
//                   $WIMWV,345,R,011.63,N,A*0A
//                 $WIMWV,300,R,2.5,N,AK*72
//                  $WIMWV,5,R,14.8N,A*29


#include <ArduinoTrace.h>;
#include <stdio.h>;
#include <string.h>;
char nmea0183_checksum(char* nmea_data);
void mwv_to_vwr();
bool inchar();
void inputread();
void storechar();
void copy_sentence_out();
bool compstr(char* str1, char* str2);


char sentence_type[4];
bool sentence_complete = false;
char tx_reqd[6];
const int maxchar = 49;
char sentence_in[maxchar + 1];
char sentence_out[maxchar + 1];
char rd;
const char valid_sentences[25] = "XTE,RCA,MWV,RCB,APB,RMC,";
const char startChar = '$';
const char endChar = '\n';
char readableChar = ' ';
const char comma = ',';

int idx = 0;
int ix = 0;
int iix = 0;
int exitix;
bool required_data = false;
char wind_tack[2];
char wind_angle[4];
char wind_speed[8];
int wind_angle_int = "";
bool receiving_data = false;
int crc;
char nullstring[3] = "";
char checkchar[3] = { nullstring };         //passing result out from checksum function
char calculated_check[3] = { nullstring };  //calculated from input string
char actual_check[3] = { nullstring };      //extracted from input string for comparison

void setup() {

  Serial.begin(9600);
  receiving_data = false;
  sentence_complete = false;
  sentence_in[0] = 0;
}

/* The main loop will read the incoming serial stream, (starting with a $ and ending in a newline)
 until the line is complete, and will then extract the sentence type to compare with the required 
 sentence types. The checksum, immediately following an asterisk (*) of the input sentence will be
 checked against the calculated checksum. If they match the data is valid. Then if the sentence type is 
 MWV, the data is translated into VWR required by the autopilot.*/



void loop() {

  //read serial input
  inputread();
  if (sentence_complete) {  //If not, wait for more characters and repeat input read until it is

    bool required_data = true;  //Flag to indicate sentence is to be transmitted, not dumped

    //Calculate check character for incoming sentence
    nmea0183_checksum(sentence_in);
    calculated_check[0] = checkchar[0];  //remember what checksum should be if not corrupted
    calculated_check[1] = checkchar[1];
    //look for check character in incoming sentence, following *
    //actual_check = "";
    for (ix = 0; ix <= (strlen(sentence_in) - 1); ix++) {
      if (sentence_in[ix] == '*') {
        actual_check[0] = sentence_in[ix + 1];
        actual_check[1] = sentence_in[ix + 2];
        break;
      }
    }
    DUMP(calculated_check);

    if (strcmp(actual_check, calculated_check) == 0) {

      // Look for valid sentences to be processed

      //look for required sentences
      for (int ix = 3; ix <= 5; ix++) {
        sentence_type[ix - 3] = sentence_in[ix];
      }
      sentence_type[3] = 0;

      required_data = false;  //data validation flag

      /* 
      Now look at each 3 character comma separated pair in valid_sentences, and compare against sentence_type
      that has been received in the input buffer
      */
      int valid_stringl = strlen(valid_sentences);
      bool exitflag = false;


      for (ix = 0; ((ix <= valid_stringl - 4) & (exitflag == false)); ix = ix + 4) {  //Is data one of the sentences reqd for autopilot?
        for (iix = 0; iix <= 2; iix++) {
          tx_reqd[iix] = valid_sentences[ix + iix];
          tx_reqd[iix + 1] = 0;
        }

        if (strcmp(tx_reqd, sentence_type) == 0) {
          required_data = true;  //Matches sentence so pass through
          exitflag = true;
        } else {  //ignore
        }
      }


      if (strcmp(sentence_type, "MWV") == 0) {
        mwv_to_vwr();
      } else {
        copy_sentence_out();
      }

      if (required_data) {
        Serial.println(sentence_out);
      } else {
        DUMP(actual_check);
        DUMP(calculated_check);
      }
    }
  }
  sentence_complete = false;
  required_data = false;

  //delay(10);
}

void mwv_to_vwr() {
  bool done = false;
  int exitix;
  for (ix = 7; (ix <= 9); ix++) {
    if (sentence_in[ix] != comma) {
      wind_angle[ix - 7] = sentence_in[ix];
      wind_angle[ix - 6] = 0;
    } else {
      break;
    }
  }
  wind_angle_int = atoi(wind_angle);

  //Skip over the next field, not required, and look for the comma
  exitix = ix;
  for (ix = exitix + 1; ix <= exitix + 2; ix++) {

    if (sentence_in[ix] == comma) {
      break;
    }
  }
  exitix = ix;
  for (ix = exitix + 1; (ix <= exitix + 5); ix++) {
    wind_speed[ix - exitix - 1] = sentence_in[ix];
    if (sentence_in[ix] == comma) {
      break;
    }
  }
  if (wind_angle_int > 180) {
    wind_angle_int = 360 - wind_angle_int;
    wind_tack[0] = 'L';
  } else {
    wind_tack[0] = 'R';
  }
  wind_tack[1] = 0;
  for (int i = 0; i <= 2; i++) {
    wind_angle[i] = '0';
  }
  wind_angle[3] = 0;
  if (wind_angle_int >= 100) {
    wind_angle[0] = '1';
    wind_angle_int = wind_angle_int - 100;
  }
  int tens = wind_angle_int / 10;
  wind_angle[1] = '0' + tens;
  wind_angle[2] = wind_angle_int % 10 + '0';
  //wind_speed[ix - exitix - 1] = 0;

  // Format output
  sentence_out[0] = 0;
  strcat(sentence_out, "$WWVWR,");
  strcat(sentence_out, wind_angle);
  strcat(sentence_out, ",");
  strcat(sentence_out, wind_tack);
  strcat(sentence_out, ",");
  strcat(sentence_out, wind_speed);
  strcat(sentence_out, ",N,,,,,*");
  nmea0183_checksum(sentence_out);
  strcat(sentence_out, checkchar);
}

void inputread() {
  if (Serial.available() > 0) {
    rd = Serial.read();
    switch (rd) {

      case startChar:
        idx = 0;  //$ encountered, start with empty buffer and store $
        receiving_data = true;
        storechar();
        break;

      case endChar:
        sentence_complete = true;
        receiving_data = false;
        break;

      default:

        if ((rd >= readableChar) && receiving_data) {
          storechar();
        }
    }
  }
}
// Save character if safe and valid part of message
void storechar() {
  if (idx < maxchar && receiving_data) {
    sentence_in[idx] = rd;
    idx++;
    sentence_in[idx] = 0;
  } else {
    Serial.println("Buffer overflow");
  }
}

//calculate checksum
char nmea0183_checksum(char* nmea_data) {
  int checksum = 0;
  int i;

  // ignore the first $ sign,  no checksum in sentence
  for (i = 1; i < (strlen(nmea_data) - 2); i++) {
    //DUMP(i);
    //DUMP(nmea_data[i]);
    if (nmea_data[i] == '*') {
      break;
    }
    checksum ^= nmea_data[i];
  }

  byte lowNibble = checksum & 0x0F;          // take the 4 lowest bits
  byte highNibble = (checksum >> 4) & 0x0F;  // take the 4 highest bits

  if (lowNibble >= 10) {
    lowNibble += 55;
  } else {
    lowNibble += 48;
  }
  highNibble += 48;

  checkchar[0] = highNibble;
  checkchar[1] = lowNibble;
  checkchar[2] = 0;
}

void copy_sentence_out() {
  strcpy(sentence_out, sentence_in);
}

Thanks in anticipation, and please don't tell me to RTFM, as I have been trying!

John

what kind of Arduino do you have?

On a typical UNO, pin 0 is Serial Rx and pin 1 is Serial Tx. They are connected also the the USB port.

➜ on which pins do you actually get your data from and on which pins do you need to send the data out? could you clarify the circuit and the modules connected to your Arduino?

I would suggest to study Serial Input Basics to handle the Serial input.

PS/ The Marine Vectors Wind format (your MVW) is not as standardised or widely documented as formats like NMEA, which means its payload structure can vary. Do you have the actual documentation for your module?

My personal thought would be to lose the Strings, and learn how to use the c-string (char) functions.

The String libraries use them anyway, so you’re just wasting cpu’s cycles unnecessarily.

1 Like

Hi,

Thanks.

I would use a Nano, and yes, you are right about the numbering of pins. I was proposing to use 0 for receive, and 1 to output, as there is no protocol involved, just an input stream and an output stream. If that doesn't work for some reason i will have to use another pin.

The input is from a commercial multiplexer with an RS422 output, and the output would ordinarily be the same. I may have to insert some signal conditioning circuits, but hope to get away without. The requirement comes from most wind instruments now using MVW. New Raymarine displays recognise it. However the design of the Raymarine tiller pilot is old and uses VWR that is supposedly obsolete. Required to maintain a course relative to apparent wind.

You do need an RS422 adaptor to connect to the serial pins of an Arduino because RS422 uses "differential signaling" with two pairs of wires (one for transmit and one for receive) and operates at possibly higher voltage levels (typically ±5V to ±12V), while the Arduino's serial pins use single-ended UART communication at lower TTL voltage levels (usually 0 to 5V or 0 to 3.3V). The adaptor converts these different voltage levels and signal types to ensure proper communication between devices.

Such ready made modules exists and are not expensive - something like that

https://www.aliexpress.us/item/3256807046108858.html

Thanks. I have built RS422 converter circuits before costing a few pence if I need to. I am concentrating first on getting some working code. I have log files with data from the actual output from the multiplexer.

+1 on avoiding Strings. Especially on AVR-based Arduinos, they cause program crashes due to poor memory management.

I strong advise to use the C/C++ C-string library instead.

We can help if you share some data. what’s the real frame format you get as input and how do you want to rewrite it. Be specific and give us some clear examples.

You also mentioned loosing sometimes a byte, do you know the cause ?

I will assume you mean a "classic" Nano V3, which is a 5V Arduino based on ATmega328 chip. (These days, there are many Arduino boards which use the 'Nano' name. Some are 5V, some are 3.3V, some are based on AVR chips and some on ARM or ESP chips. Things have got very confusing!)

I would recommend switching to Pro Micro (not to be confused with Pro Mini), because the Pro Micro has a second available hardware serial port.

On Nano, there is only one hardware serial port (pins 0, 1), and this is connected to the Nano's on-board USB-Serial adapter chip, for uploading code from your PC/laptop to the Arduino and for debugging your code.

While it might be possible to use this hardware serial port for connection to the RS422 multiplexer, you can't connect it to both the multiplexer and the PC/laptop at the same time. That means continual plugging and unplugging between the PC and the multiplexer during the development process, and no easy way to debug the code.

If you use other pins to connect the Nano to the multiplexer, you are forced to use Software Serial, which is inferior to hardware serial in various ways and can result in loss of data unless the code is very carefully written.

The Pro Micro has an extra serial port which is dedicated to USB communications with the PC and all code uploading and debugging is taken care of by that port. This means the other hardware serial port on pins 0, 1 is free to use for your purposes.

Just an update. Thanks to those who have taken an interest, and particularly the advice to use Char arrays rather than strings. It is of course MWV we are talking about.

I am making good progress with the learning curve, but have come to realise from testing with genuine data that this is comma separated traffic rather than fixed format, so I have to extract the data I need dynamically so have a bit more work to do.

can you share some frames ? we could help

Hi,

Do you mean test data?

No time at the moment, but will try to get back to you later. ..

John

yes, the frames you are using to write your parser

Hi again,

I am still not clear what you mean by 'frames' - a new one to me.

The fields need to be parsed using the , separators, as they are not consistent in presentation.

I can attach a typical log file of data and the spec for MWV sentences. This comes from the multiplexer USB serial output I believe, but it is a while since I took them off. They are far from clean data, many of the $WIMWV sentences, notable CRLF's missing. The data appears in the OpenCPN navigation application (via wifi from the mux) and appears there to have been cleaned up - invalid sentences no doubt dropped. Corruptions may be due to mux overload, but blacklisting is possible to a certain extent. I will have to look into this on site.

The RS422 output from the mux will drive the rx input to the Arduino.. have yet to detail that, and the output will drive some circuitry to create RS422 signals at 5v into the autopilot.

When I have confidence in the checksum code, I will use it to compare with the input, and drop any sentences that do not tally. (At the moment it appears to be generating a null string.) That logic is not there yet. I will also pass through other sentences as needed.

The code has trace/dump still within it, and as yet is work in progress, also attached

.

(attachments)

WMT700 NMEA MWV data message • WMT700 Series User Guide • Reader • Product documentation portal.pdf (186 KB)
nmea_vwr_conversion_29240903.ino (3.61 KB)
putty_160402.log (72.1 KB)

Your log file would hold many frames, sentences, payloads … many names can be used ➜ it means a series of value, comma separated in your case as you mention, that forms a complete entry.

So yes share the log file.

See attachments above!

I had missed it

so your input is like

.70,,,,,43,16,6,64,41*4D
$GPG$WIMWV,1
$HCHDG,,E*37
$WIMWV,6,R,7.4,N,A*38
$WIMWV,5,R,7.4,NVWVLW,18IMWV,5,
$WIMWV3,N,A*3C,N*04
$IIVHW,,T,162.00,8,R,7.2,5,R,7.,,K*4E
162.00,,IIVHW,,T,162.00.00,,,,*1,N,A*32
$VWVHW.70,M,,N,,K*4A
B
$GPG,4,6,7,920,26,2917,32,204
$WIMWV,13,R,7.1,N,A*09
$II,T,,0.00,N*76
$WIMWV,1N,A*0E
$WIMWV,13,R,7.1A*22
$T804,N,49WIMWV,381.70,,,,5.9,N,,N,,N*6E
$GPVDR,0.66,T,,IIVHW,,$IIVHW,0,M,,N,,1,48,5,1,292,26,6,59,209,42*71
$GPGSV,4,2,15,7,39,155,43,9,77,69,50,11,50,291,43,16,6,64,39
$WIMWV,2,R,13.5,N,A*08
$WIMWV,5,R,13.8,N,A*02
$TIROT,0.00,A*0B
$TM,,N,,K4.1,N,A6,T,,M,.8,N,A*0,,,,*71
$IIVHW,,T,161.50,M,,NN,A*09
$WIMWV,5,R,14.8N,A*05
$WIMWV,6,N,A*06
$GPGSV,4,3,15,19,3,217,33,20,28,296,40,26,6,30,41,29,$HCHDG,,,,0.66,E*37
$WIMWV,8,R,14.9,N,N,,K*48
$HCHDG161.50,M,,N,,K*48
$HCHDG,161.50,,,,*71
$IIVHW,,T,161.50,M,,N,,K*48
$HCHDG4.9,N,A,,N,,N*0,,,,*730.93*264,3,15,1,16.9,N,70,,,,*73
$IIVHW,,T,16,N,,K*48TIROT,-1C
$WIMWA*3A
$HCHDG,,,,,,T,160.80,M,,N,,K*44
G,160.80,,,,*7D
,9,R,15.
$TIROT,3.70,A*0F
$TI0,A*0F
,R,15.81,N,A*08
$VWVH
$TIROT
$WIMWV
$GPGSV,4,2,15,7,40,157,69,52,11,50,291,43,16,6,64,40*40
$GPGSV,4,3,15,19,3,20,28,296,40,26,6,30,36,29,6,340,26*7F
$GPGSV,4,4,15R,15.0,N61.80,MR,0.66,T,,0.00,NIMWV,12,,A*35
,,K*45
HDG,161.72
$IIV.2,N,A*325,R,13.2,N,A*3A
$GPGSA,A,2,3,4,6,7,9,,9,76,68,50,11,50,291,45,16,6,64,40*44
$GPGSV,4,3,15
$HCHDG,161.60,,,,*72
N,A*0F
F
$HCH1.5,N,A*HW,,T,161.40,M,,N,,K*49
$HCHD
$IIVHW,,T,161.40,M,,N,,K*49
$HCHDG,161.40,,,,*70
$WIMWV,35M,0,N,,K4,39*4B
$GPGSV,4,3,15,19,3,21VHW,,T,161.40,M,,N,,K*49
$HCHD0,,,,*70
...

and your documentation describes the format as

lines end with CR LF (when received correctly) and start with $ and you want to extract and parse all the lines starting with $WIMWV

is that correct ?

Hi

I am moving on with the sketch itself. I will attach my present non-compiling version to bring you up to date.

Each NMEA0183 sentence should begin with a $, contain a check character being the exclusive or of all characters between, but not including, the $ and the *, and are terminated with a CRLF.

My input is all working. I am currently wrestling with a type mismatch between the check character and the data in the input buffer 'sentence_in'. There was a typo there that resulted in my getting clean compiles, but not the right check character. I have changed the - to an = and that is stopping me at the moment. Less than clear understanding of pointers etc. Then I need to work on the comma delimited fields.

The only data I need out of an MWV sentence is actually the bearing - no problem with this hitherto*, but should include wind speed in VWR sentence output. This is a floating number, and seems to vary in length. * But also, I see in the data that the leading zeros of the direction (000 - 360) in the MWV can also be dropped, so need to take account of that by working on the comma delimiters.

(attachments)

nmea_vwr_conversion_29240903_copy_20240905145329.ino (4.38 KB)

I'm reading on my phone, please post code right here using code tags.

Please post your code properly. In-line with Code Tags, not as an attachment.