Serial Interpreter Library for g-codes or own defined space separated codes

Hello everyone,

I wrote a library in C which uses HardwareSerial to link serial inputs to functions, which I want to share with anyone who can have a use to it.
It's an interpreter which can interpret your own defined codes and read the values of your inputs.
On my github I made an example for g-codes. F.e. "G01 X100 Y200\n" will store the values of G, X and Y in a struct which will be processed at a new line in an own function.
Using this you can implement a list of g-codes for your machine in a very simple way.
You can use it if you want to make a cnc-machine or a robot which you want to control using g-codes or own codes separated by spaces or new lines.

You can find this library here: GitHub - SebisCodes/SebisSerialInterpreter: A library to run functions by your g-code similar codes

Strings and blocking behavior would prevent me from even trying to use it.

What do you mean with blocking behavior?

A lot of while loops:

while (fl->next != NULL)
    {

With this code your library is unlikely to be able to do anything else at the same time as processing the g-code. This is called blocking behavior

I think this particular snippet walks through a (short) linked list. I would be more worried about this:

while (this->hSerial->available() <= 0 && endlessLoop);

I agree, I chose the wrong example. I meant that the library contains many such cycles.

You can simply turn it off by calling serialRead(false); instead of serialRead(true); and do your code and check later again if there is serial data.
I need to check if I can make the iteration through the list more efficient, thanks for this reply.

If you have nothing to read in Serial buffer - your function shouldn't wait for new bytes, it should exit immediately and free the controller for other tasks.

No, this should not be configured by some switches in the code, otherwise sooner or later it will all hang.

Thank you for the effort & sharing.

Like many software projects, I am certain you are pleased with your project and have tested it with your use case(s). You have succeeded in resolving your specific needs.

Once a thousand eyes start staring away at your project, expect critics to find numerous faults that may have never been considered in your requirements and critics can be vocal. Hopefully, you will just weather the criticism and take the opportunity to validate their concerns; it is your code and only you can decide how much to broaden the appeal to others.

Maybe some of our members that work with g-code will utilize GitHub and provide pull-requests.

Most people will expect this to be the default when using a library, they also want to blink LEDs and stuff.

You could perhaps copy the incoming data into a buffer and process it whenever the end of line character(s) is/are found. This way the library never waits for incoming data.

[edit]

I use the following functions in one of my projects. Maybe they are of use to you.

/*! Thread safe, memory safe non blocking read until delimiter is found.
 *
 * \tparam bufSize Size of receiving buffer.
 *
 * \param[in,out] stream Input stream.
 * \param[out] buf Receiving buffer.
 * \param[in] delim Delimiter.
 * \param[in,out] bufIdx Buffer index.
 * \param[in,out] delimIdx Delimiter index.
 *
 * \return true when delimiter is found or the buffer is full, false otherwise.
 */
template <size_t bufSize>
bool readUntil_r(
    Stream& stream, char (&buf)[bufSize], char const* delim,
    size_t& bufIdx, size_t& delimIdx) {
  for (; bufIdx < bufSize and delim[delimIdx]; bufIdx++) {
    if (not stream.available()) {
      return false;
    }
    buf[bufIdx] = stream.read();
    if (buf[bufIdx] == delim[delimIdx]) {
      delimIdx++;
    }
    else {
      delimIdx = 0;
    }
  }
  for (; bufIdx < bufSize; bufIdx++) {
    buf[bufIdx] = 0;
  }
  bufIdx = 0;
  delimIdx = 0;
  return true;
}

/*! Memory safe non blocking read until delimiter is found.
 *
 * \tparam bufSize Size of receiving buffer.
 *
 * \param[in,out] stream Input stream.
 * \param[out] buf Receiving buffer.
 * \param[in] delim Delimiter.
 *
 * \return true when delimiter is found or the buffer is full, false otherwise.
 */
template <size_t bufSize>
bool readUntil(Stream& stream, char (&buf)[bufSize], char const* delim) {
  static size_t bufIdx {0};
  static size_t delimIdx {0};
  return readUntil_r(stream, buf, delim, bufIdx, delimIdx);
}

Thank you, I just found a bug I fixed in serialRead().
I read everything from the Serial and checked then the last char, which means if there is an input with multiple new lines, it had been passed.
Now the new line is checked after every read char to don't miss any functions.

Ladyada did this rather cleverly in the old version of her GPS library.

/***********************************
This is our GPS library

Adafruit invests time and resources providing this open source code,
please support Adafruit and open-source hardware by purchasing
products from Adafruit!

Written by Limor Fried/Ladyada for Adafruit Industries.
BSD license, check license.txt for more information
All text above must be included in any redistribution
****************************************/
#include "Adafruit_GPS.h"

// how long are max NMEA lines to parse?
#define MAXLINELENGTH 120

// we double buffer: read one line in and leave one for the main program
volatile char line1[MAXLINELENGTH];
volatile char line2[MAXLINELENGTH];
// our index into filling the current line
volatile uint8_t lineidx=0;
// pointers to the double buffers
volatile char *currentline;
volatile char *lastline;
volatile boolean recvdflag;
volatile boolean inStandbyMode;


boolean Adafruit_GPS::parse(char *nmea) {
  // do checksum check
  // first look if we even have one
  if (nmea[strlen(nmea)-4] == '*') {
    uint16_t sum = parseHex(nmea[strlen(nmea)-3]) * 16;
    sum += parseHex(nmea[strlen(nmea)-2]);
    
    // check checksum 
    for (uint8_t i=1; i < (strlen(nmea)-4); i++) {
      sum ^= nmea[i];
    }
    
    if (sum != 0) {
      // bad checksum :(
      //return false;
    }
  }

/*
  // look for a few common sentences
  if (strstr(nmea, "$GPGGA")) {
    // found GGA
    char *p = nmea;
    // get time
    p = strchr(p, ',')+1;
    float timef = atof(p);
    uint32_t time = timef;
    hour = time / 10000;
    minute = (time % 10000) / 100;
    seconds = (time % 100);

    milliseconds = fmod(timef, 1.0) * 1000;
    return true;
  }
*/

  if (strstr(nmea, "$GPRMC")) {
   // found RMC
    char *p = nmea;

    // get time
    p = strchr(p, ',')+1;
    float timef = atof(p);
    uint32_t time = timef;
    hour = time / 10000;
    minute = (time % 10000) / 100;
    seconds = (time % 100);

    milliseconds = fmod(timef, 1.0) * 1000;
    p = strchr(p, ',')+1;  // A/V?
    p = strchr(p, ',')+1;  // lat
    p = strchr(p, ',')+1;  // N/S?
    p = strchr(p, ',')+1;  // lon
    p = strchr(p, ',')+1;  // E/W?
    p = strchr(p, ',')+1;  // speed
    p = strchr(p, ',')+1;  // angle

    p = strchr(p, ',')+1;
    uint32_t fulldate = atof(p);
    day = fulldate / 10000;
    month = (fulldate % 10000) / 100;
    year = (fulldate % 100);
    // we dont parse the remaining, yet!
    return true;
  }

  return false;
}

char Adafruit_GPS::read(void) {
  char c = 0;
  
  if (paused) return c;

    if(!gpsSwSerial->available()) return c;
    c = gpsSwSerial->read();

  //Serial.print(c);

  if (c == '$') {
    currentline[lineidx] = 0;
    lineidx = 0;
  }

  if (c == '\n') {
    currentline[lineidx] = 0;

    if (currentline == line1) {
      currentline = line2;
      lastline = line1;
    } else {
      currentline = line1;
      lastline = line2;
    }

    //Serial.println("----");
    //Serial.println((char *)lastline);
    //Serial.println("----");
    lineidx = 0;
    recvdflag = true;
  }

  currentline[lineidx++] = c;
  if (lineidx >= MAXLINELENGTH)
    lineidx = MAXLINELENGTH-1;

  return c;
}

Adafruit_GPS::Adafruit_GPS(SoftwareSerial *ser)
{
  common_init();     // Set everything to common state, then...
  gpsSwSerial = ser; // ...override gpsSwSerial with value passed.
}

// Initialization code used by all constructor types
void Adafruit_GPS::common_init(void) {
  gpsSwSerial = NULL; // Set both to NULL, then override correct
  recvdflag   = false;
  paused      = false;
  lineidx     = 0;
  currentline = line1;
  lastline    = line2;
  hour = minute = seconds = year = month = day = fixquality = satellites = 0; // uint8_t
  milliseconds = 0; // uint16_t
}

void Adafruit_GPS::begin(uint16_t baud) {
  if(gpsSwSerial) gpsSwSerial->begin(baud);
  delay(10);
}

boolean Adafruit_GPS::newNMEAreceived(void) {
  return recvdflag;
}

void Adafruit_GPS::pause(boolean p) {
  paused = p;
}

char *Adafruit_GPS::lastNMEA(void) {
  recvdflag = false;
  return (char *)lastline;
}

// read a Hex value and return the decimal equivalent
uint8_t Adafruit_GPS::parseHex(char c) {
    if (c < '0')
      return 0;
    if (c <= '9')
      return c - '0';
    if (c < 'A')
       return 0;
    if (c <= 'F')
       return (c - 'A')+10;
}

boolean Adafruit_GPS::waitForSentence(char *wait4me, uint8_t max) {
  char str[20];
  uint8_t i=0;
  while (i < max) {
    if (newNMEAreceived()) { 
      char *nmea = lastNMEA();
      strncpy(str, nmea, 20);
      str[19] = 0;
      i++;

      if (strstr(str, wait4me))
	return true;
    }
  }

  return false;
}



I see your point, this is way more efficient to use char arrays than use stirngs, because if I add a char to a string, it creates a completely new char array which will be completely overwritten.
I read and store the input chars from serial in a kind of buffer, but my buffer is a string instead of a char array which has the benefit of beeing dynamic for longer codes, becuase the limit is very very far away from reachable if this is important for anyone.
I could implement a second serialRead method which uses char arrays, but is limited in length like your solution, but you're right, this will be much more efficient, thank you.

The point was more that you could drop the option for having blocking behaviour altogether. Using a String is not a bad solution in my opinion, provided your board has enough memory.

I presume the g-code messages in question are limited in length anyway?

Now I've got your point, thanks for that.
I extended my function by a boolean which you can use to prevent this blocking behavior.
Using this boolean it'll only read char by char no matter how much chars there are available.
What do you think about that?

I'm not sure if there is any length limitation for g-codes. For sure they'll never reach 120 chars per line, but this library is not only for g-codes, you can also link functions to own codes or words, it's very dynamic in use.

Nice job.

You could consider not offering blocking behaviour in the first place, or do you have a specific use case in mind that would benefit from it?

That will work nicely I think. Alternatively, you could read all available data in one go. As long as you do not try to read when nothing is available, there should be no blocking behaviour.

You do know that the serial buffer itself is limited in length? I think a commonly used default is 64 bytes.

Tanks! :slight_smile:
Yes, the reason is that the first boolean is creating an own loop inside the library which causes a blocking of all other procedures outside the library (f.e. an led-hearbeat; btw. I could implement an library internal heartbeat). This may be used if you have no other code at all which you want to use except the functions which are linked to the keywords, so the second boolean says if you don't use the library loop, you can choose if you want to have blocking behavior (read all data from serial) or only read char by char.
I thought about using the getString() method, but I had bad experience with it in the past and this won't work with my code, because new line chars will be passed if the last char isn't a new line char.
I need to rewrite my library for that, but this works fine for the most cases as it is and if someone really needs performance I could try to rewrite it or create a new method.

I thought there is a buffer with a length, but I guess I'll never reach it if I use the library internal loop, because the processing of the readout should be faster than 115200 bauds I guess, but that's only a guess, I didn't check it at all, but that's a good point.