Hello, I have a text document that only contains this,
L,R,D',B,U,D,F',R,B',L',B2,R,F2,D2,F,L',U2,L,U2,L,B2,L',B2,L2,U2,L2,R2,U2,B2,U2
What I need to do is find a way to read it.
I need to first read L, then call a function that I created that will activate to FunctionL
Then R with function FunctionR
Then D' with function FunctionD'
so on so forth
I am normally used to matlab, so some of my notation is messed up, any help is appreciated!
so it seems you have to parse a comma separated list.
you can use strtok() to do so and then for each token (L, R, D, B2, ....) you could compare with a list of known tokens and call an attached function (stored in an array to make your life easy)
here is an example
char dataToScan[] = "L,R,L,R,B2,U2,D',B2,U2,SHARED1,SHARED2";
void func_L(const char* s) {Serial.print(F("in Function ")); Serial.println(s);}
void func_R(const char* s) {Serial.print(F("in Function ")); Serial.println(s);}
void func_Dp(const char* s) {Serial.print(F("in Function ")); Serial.println(s);}
void func_B2(const char* s) {Serial.print(F("in Function ")); Serial.println(s);}
void func_U2(const char* s) {Serial.print(F("in Function ")); Serial.println(s);}
void func_SHARED(const char* s) {Serial.print(F("in Shared Function, input was ")); Serial.println(s);}
struct {
const char* label;
void (*func)(const char*);
} keys[] = {
{"L", func_L},
{"R", func_R},
{"D'", func_Dp},
{"B2", func_B2},
{"U2", func_U2},
{"SHARED1", func_SHARED},
{"SHARED2", func_SHARED}
};
const size_t nbKeys = sizeof keys / sizeof keys[0];
bool findAndExecute(const char* s)
{
bool success = false;
for (size_t i = 0; i < nbKeys; i++)
if (!strcmp(s, keys[i].label)) {
keys[i].func(s);
success = true;
break;
}
return success;
}
void setup() {
Serial.begin(115200);
Serial.print(F("\n\nparsing : ")); Serial.println(dataToScan);
char* ptr = strtok(dataToScan, ",");
while (ptr) {
findAndExecute(ptr);
ptr = strtok(NULL, ",");
}
}
void loop() {}
Serial Monitor (@ 115200 bauds) will show
[color=purple]
parsing : L,R,L,R,B2,U2,D',B2,U2,SHARED1,SHARED2
in Function L
in Function R
in Function L
in Function R
in Function B2
in Function U2
in Function D'
in Function B2
in Function U2
in Shared Function, input was SHARED1
in Shared Function, input was SHARED2
[/color]
My Arduino Serial I/O for the Real World tutorial has a detailed CSV (comma separated values) parsing example,
When you say 'document' where is the data stored?
drmpf:
My Arduino Serial I/O for the Real World tutorial has a detailed CSV (comma separated values) parsing example,When you say 'document' where is the data stored?
The data is stored in the same folder as my code. It is a notepad document called transition.txt. Also important to note, the arduino will be connected to my computer the whole time.
OK you have to get it into your Arduino so how.
Choices are
i) read in via Serial
ii) define in the code as a
const char myData[] = "L,R,D',B,U,D,F',R,B',L',B2,R,F2,D2,F,L',U2,L,U2,L,B2,L',B2,L2,U2,L2,R2,U2,B2,U2";
iii) put it on an SD card and read it from there (via add on SD reader and extra software)
If it is not going to change ii) code is easiest
If you want to 'control' it, use Serial input and check out my CSV GPS parsing example.
Also important to note, the arduino will be connected to my computer the whole time.
what’s the purpose of the arduino? Are you using functions to drive some equipment ?
J-M-L:
what’s the purpose of the arduino? Are you using functions to drive some equipment ?
Yea, I am controlling 6 stepper motors drivers with it. I have a function written for what I want the drivers to output (either a 90 degree turn clockwise, counter clockwise, or a 180 degree turn). This is done for each servo. Basically if the character array states L, it will tell the left servo to do a 90 degree turn clockwise. If it says R', the right servo will move 90 counter clockwise. If it says U2, the top servo will move 180 degrees. Only issue is transmitting that one character array.
If you have any ideas please let me know
I would do it all in MATLAB, but it is incredibly slow to control servos with.
A small script running on the computer and sending the commands to the arduino through Serial?
If you had a Wi-Fi / Ethernet board available you could also use http or ftp to fetch the orders from a know location or the arduino would offer an http web interface and you copy paste the commands there.
In a nutshell, your arduino does not have direct access to your computer’s hard drive. (Luckily, think of all the security issues!) and so you need some sort of process running on the computer to export the data to the arduino.
If you had a Wi-Fi / Ethernet board available you could also use http or ftp to fetch the orders from a know location or the arduino would offer an http web interface and you copy paste the commands there.
I second J-M-L's idea, but you might like to sort out the parsing from the Serial input first and then move on to an ESP32 or ESP8266 to do the web approach.
Parsing is already done in my example posted previously, it’s trivial
char* ptr = strtok(dataToScan, ","); // extract first token
while (ptr) {
findAndExecute(ptr); // deal with token
ptr = strtok(NULL, ","); // grab next token if there is one
}
The complexity if there is a process pushing the commands through Serial is that you don’t want to push too fast and saturate buffers as there are motor movements which can take time and so possibly some throttling will be needed, which would require some sort of 2 way communication.
Grabbing the full command list (if it fits in memory) before starting execution makes it easier. I would suggest to study Serial Input Basics to handle this if pushed through Serial (just send the commands with a line feed at the end)
Coming from Matlab you will expect strings to be well behaved.
To get similar behaviour in Arduino, you need to use Arduino Strings
Avoid low level c-string methods (even though they have similar names to the matlab fns) as in Arduino (C/C++) they work on fixed length char[] not Strings. They are very pone to coding errors, so much so that Microsoft has banded their use and text books have been written on why they should not be used.
Here is some sample code that reads a line of comands and executes them.
Because Strings are being used here, it does not matter if the length of your command line increases. The String inputLine will just expand as necessary, as in matlab. The strInput_RESERVE is just a hint.
// sample input
// L,R,D',B,U,D,F',R,B',L',B2,R,F2,D2,F,L',U2,L,U2,L,B2,L',B2,L2,U2,L2,R2,U2,B2,U2
// terminated with a NewLine char '\n' or CR NL
String inputLine;
unsigned int strInput_RESERVE = 120; // an estimate of the input line length no need to be exact
// stopC is the char to stop at
// returns true when get stopC and input contains the input up to then
// else return false
bool readStrUntil(char stopC, String& input) {
while (Serial.available()) {
char c = Serial.read();
if (c == stopC) {
return true; // input has char upto stopC
}
// else
input += c;
if (input.length() >= strInput_RESERVE) {
// prevent re-allocation and fragmenation
return true; // input has char upto stopC
}
}
return false;
}
void runL() {
Serial.println("run cmd L");
}
void runR() {
Serial.println("run cmd R");
}
/// etc
void unknownCmd(String& cmd) {
Serial.print("cmd:"); Serial.print(cmd); Serial.println(" not programmed yet.");
}
void execute(String& cmd) {
cmd.trim();
if (cmd.length() == 0) {
return;
}
if (cmd == "L") {
runL();
} else if (cmd == "R") {
runR();
// . . . etc
} else {
unknownCmd(cmd);
}
}
void parseInput(String& data) { // note the String&
Serial.print(F("Data :")); Serial.println(data);
data.trim();
if (data.length() == 0) { //no data just return
return;
}
data += ','; // add a trailing , to simplify the logic
int idx = data.indexOf(',');
while (idx >= 0) { // not -1 i.e found ,
String cmd = data.substring(0, idx);
execute(cmd);
data = data.substring(idx + 1); // step over the , and remove the cmd just processed
idx = data.indexOf(',');
}
}
void setup() {
Serial.begin(115200);
for (int i = 10; i > 0; i--) {
Serial.print(i); Serial.print(' ');
delay(500);
}
Serial.println();
inputLine.reserve(strInput_RESERVE); // allow some initial space
}
void loop() {
if (readStrUntil('\n', inputLine)) { // got a line of input
parseInput(inputLine);
inputLine = ""; // clear for next input
}
}
Collecting the whole line and processing it in one go is the simplest solution and avoids execution timing problems.
In the sketch above I am sending some debug output back to the Serial. Test this code out from the Arduino IDE first.
When connected to matlab, the debug output will go back there.
The SafeString references I gave above can be used for whole line processing, but use char[] and so will not expand as needed. You can use the SafeStringReader to read a field at a time, but then you will be reading and executing 'at the same' time. It can be done but takes a bit more care. See my tutorial on Multi-tasking in Arduino
drmpf:
Coming from Matlab you will expect strings to be well behaved.
To get similar behaviour in Arduino, you need to use Arduino Strings
Avoid low level c-string methods (even though they have similar names to the matlab fns) as in Arduino (C/C++) they work on fixed length char[] not Strings. They are very pone to coding errors, so much so that Microsoft has banded their use and text books have been written on why they should not be used.Here is some sample code that reads a line of comands and executes them.
Because Strings are being used here, it does not matter if the length of your command line increases. The String inputLine will just expand as necessary, as in matlab. The strInput_RESERVE is just a hint.// sample input
// L,R,D',B,U,D,F',R,B',L',B2,R,F2,D2,F,L',U2,L,U2,L,B2,L',B2,L2,U2,L2,R2,U2,B2,U2
// terminated with a NewLine char '\n' or CR NL
String inputLine;
unsigned int strInput_RESERVE = 120; // an estimate of the input line length no need to be exact
// stopC is the char to stop at
// returns true when get stopC and input contains the input up to then
// else return false
bool readStrUntil(char stopC, String& input) {
while (Serial.available()) {
char c = Serial.read();
if (c == stopC) {
return true; // input has char upto stopC
}
// else
input += c;
if (input.length() >= strInput_RESERVE) {
// prevent re-allocation and fragmenation
return true; // input has char upto stopC
}
}
return false;
}
void runL() {
Serial.println("run cmd L");
}
void runR() {
Serial.println("run cmd R");
}
/// etc
void unknownCmd(String& cmd) {
Serial.print("cmd:"); Serial.print(cmd); Serial.println(" not programmed yet.");
}
void execute(String& cmd) {
cmd.trim();
if (cmd.length() == 0) {
return;
}
if (cmd == "L") {
runL();
} else if (cmd == "R") {
runR();
// . . . etc
} else {
unknownCmd(cmd);
}
}
void parseInput(String& data) { // note the String&
Serial.print(F("Data :")); Serial.println(data);
data.trim();
if (data.length() == 0) { //no data just return
return;
}
data += ','; // add a trailing , to simplify the logic
int idx = data.indexOf(',');
while (idx >= 0) { // not -1 i.e found ,
String cmd = data.substring(0, idx);
execute(cmd);
data = data.substring(idx + 1); // step over the , and remove the cmd just processed
idx = data.indexOf(',');
}
}
void setup() {
Serial.begin(115200);
for (int i = 10; i > 0; i--) {
Serial.print(i); Serial.print(' ');
delay(500);
}
Serial.println();
inputLine.reserve(strInput_RESERVE); // allow some initial space
}
void loop() {
if (readStrUntil('\n', inputLine)) { // got a line of input
parseInput(inputLine);
inputLine = ""; // clear for next input
}
}
Collecting the whole line and processing it in one go is the simplest solution and avoids execution timing problems. In the sketch above I am sending some debug output back to the Serial. Test this code out from the Arduino IDE first. When connected to matlab, the debug output will go back there. The SafeString references I gave above can be used for whole line processing, but use char[] and so will not expand as needed. You can use the SafeStringReader to read a field at a time, but then you will be reading and executing 'at the same' time. It can be done but takes a bit more care. See my tutorial on [Multi-tasking in Arduino](https://www.forward.com.au/pfod/ArduinoProgramming/RealTimeArduino/index.html)
Thank you all for your responses. It will help me in trying to get to my end goal. A couple of questions.
When I connect the arduino to matlab, I need to specify a COM port and baud rate. I can do this, but the COM port can only be used by one thing at a time? So I couldnt run the arduino code and the Matlab at the same time?
I am liking your idea with collecting the whole line and then processing it. How long do you think it would be to transfer a line like that. Would it be easier for me to covert it to numbers in matlab, then send it over? Once it gets the code, i need the execution to be quick, so I dont mind about waiting for it to process.
If you set the serial connexion at 1 million bauds, you’ll send a line with 500 one letter command and the commas in roughly 1/100th of a second. Is that good enough?
Parsing is fast so the slow part are your motors.
I disagree with drmpf on banning using cStrings.They have their quirks and indeed on modern operating systems with virtual memory and gigs of ram, there are better alternatives. On a small microcontroller it’s a different world and every byte of SRAM or Flash or microsecond often counts. So often they are your best friend and it’s best to know about those.
How big is the largest command you envision sending? If it’s too big for an Arduino’s memory then you’ll have to devise a different strategy to handle big frames. If it does fit, just preallocate that memory and just ensure in the code that you don’t overflow upon reception. (In your specific case using a pre-allocated String can bring a bit of peace of mind but you’ll have nevertheless to tests for memory issues)
J-M-L:
If you set the serial connexion at 1 million bauds, you’ll send a line with 500 one letter command and the commas in roughly 1/100th of a second. Is that good enough?
Parsing is fast so the slow part are your motors.I disagree with drmpf on banning using cStrings.They have their quirks and indeed on modern operating systems with virtual memory and gigs of ram, there are better alternatives. On a small microcontroller it’s a different world and every byte of SRAM or Flash or microsecond often counts. So often they are your best friend and it’s best to know about those.
How big is the largest command you envision sending? If it’s too big for an Arduino’s memory then you’ll have to devise a different strategy to handle big frames. If it does fit, just preallocate that memory and just ensure in the code that you don’t overflow upon reception. (In your specific case using a pre-allocated String can bring a bit of peace of mind but you’ll have nevertheless to tests for memory issues)
J-M-L:
If you set the serial connexion at 1 million bauds, you’ll send a line with 500 one letter command and the commas in roughly 1/100th of a second. Is that good enough?
Parsing is fast so the slow part are your motors.I disagree with drmpf on banning using cStrings.They have their quirks and indeed on modern operating systems with virtual memory and gigs of ram, there are better alternatives. On a small microcontroller it’s a different world and every byte of SRAM or Flash or microsecond often counts. So often they are your best friend and it’s best to know about those.
How big is the largest command you envision sending? If it’s too big for an Arduino’s memory then you’ll have to devise a different strategy to handle big frames. If it does fit, just preallocate that memory and just ensure in the code that you don’t overflow upon reception. (In your specific case using a pre-allocated String can bring a bit of peace of mind but you’ll have nevertheless to tests for memory issues)
Id stay the largest is going to be 40 combinations of letters. I am newer to arduino, so not sure where to start. Should I create a special character for the front and the end or the array?
This seems simple enough to just end the list by a new line
Make the buffer big enough, may be 200 bytes that should be plenty for 40 commands and commas
The code in the Serial Input Basics tutorial handles that fine and then you just parse using strtok and attach functions to tokens in an array as provided in my previous post and you should be good to go. You’ll only need to figure out how to send the command from the PC
Arduino Strings are so easy to use I would start with them and once you have that working then see what the speed is like. You can use my loopTimer class to check the loop speed. See my Multi-tasking in Arduino tutorial.
Later you can always transfer to my SafeStrings library which offer similar flexibility to Arduino Strings but use char[]'s and wraps the underlying c-string method with the coding checks a diligent programmer would add. So you get the low level speed, but you still avoid the systemic coding errors that occur using c-strings.
For the matlab serial connection, the defaults look good.
Use writeline to send the data with a LF ('\n') terminator .
Unfortunately matlab reads are blocking so you need to write code like mine above on the matlab side to check for NumBytesAvailable > 0, read a char and add it to a String.
Use datatype char for the read.
By the way, what Arduino board are you using?
Arduino Strings are essentially bullet proof on UNO and Mega2560.
drmpf:
Later you can always transfer to my SafeStrings library which offer similar flexibility to Arduino Strings but use char[]'s and wraps the underlying c-string method with the coding checks a diligent programmer would add. So you get the low level speed, but you still avoid the systemic coding errors that occur using c-strings.
there is nothing so systemic about c-strings, you are just saying you just need to be a diligent programmer to not face any problem.
Receiving bytes in an array until getting a special byte and being careful not to write past the end of the array is all what it takes in OP’s case. Does not seem out of reach for any programmer.
In a nutshell, for OP’s ask and expectations, there is no need for adding clutter with non standard libraries. What’s built in is already plenty and efficient, memory wise and speed wise.
here is sample code receiving from Serial (and outputting some messages back to Serial) at 115200 bauds
const size_t maxCommandNumber = 50;
const size_t maxCommandSize = 3 * maxCommandNumber; // if commands fit on 2 characters and a comma, this is enough for maxCommandNumber commands
char receivedChars[maxCommandSize + 1]; // +1 for trailing null char required by cStrings
const char endMarker = '\n';
void func_L(const char* s) {Serial.print(F("in Function ")); Serial.println(s);}
void func_R(const char* s) {Serial.print(F("in Function ")); Serial.println(s);}
void func_Dp(const char* s) {Serial.print(F("in Function ")); Serial.println(s);}
void func_B2(const char* s) {Serial.print(F("in Function ")); Serial.println(s);}
void func_U2(const char* s) {Serial.print(F("in Function ")); Serial.println(s);}
void func_unknown(const char* s) {Serial.print(F("Unknown commmand: ")); Serial.println(s);}
struct {
const char* label;
void (*func)(const char*);
} keys[] = {
{"L", func_L},
{"R", func_R},
{"D'", func_Dp},
{"B2", func_B2},
{"U2", func_U2},
};
const size_t nbKeys = sizeof keys / sizeof keys[0];
bool findAndExecute(const char* s)
{
bool success = false;
for (size_t i = 0; i < nbKeys; i++)
if (!strcmp(s, keys[i].label)) {
keys[i].func(s);
success = true;
break;
}
if (!success) func_unknown(s);
return success;
}
bool commandAvailable()
{
static size_t commandIndex = 0;
static bool awaitEndMarkerError = false;
int incomingByte = Serial.read();
bool commandComplete = false;
if (incomingByte != -1) { // read returns -1 if there is nothing to read (faster than using available)
if (awaitEndMarkerError) {
if (incomingByte == endMarker) {
awaitEndMarkerError = false;
commandIndex = 0;
receivedChars[0] = '\0'; // discard what was there
commandComplete = true; // if we want to still notify the parser we got a command (here will be empty)
}
} else {
char incomingCharacter = incomingByte; // grab LSB as a char
if (incomingByte != endMarker) {
if (commandIndex < maxCommandSize) {
if (!isspace(incomingCharacter)) { // discard all spaces. could also add a test to only keep A-Z and ' and 2, whatever is legit in the command language
receivedChars[commandIndex++] = incomingCharacter;
receivedChars[commandIndex] = '\0'; // always maintain a correct cString
}
} else {
Serial.println(F("ERROR: Command line too big. Discarded.\n\t-> increase maxCommandNumber or send shorter command lines."));
awaitEndMarkerError = true;
}
} else {
commandIndex = 0;
commandComplete = true;
}
}
}
return commandComplete;
}
bool handleCommand()
{
bool success = true;
const char* separators = ",";
Serial.print(F("\nparsing : ")); Serial.println(receivedChars);
char* ptr = strtok(receivedChars, separators);
if (*ptr == '\0') Serial.println(F("Error: empty command"));
else
while (ptr) {
success &= findAndExecute(ptr);
ptr = strtok(NULL, separators);
}
return success;
}
void setup() {
Serial.begin(115200);
}
void loop() {
if (commandAvailable())
if (! handleCommand())
Serial.println(F("Some commands were not recognized"));
}
The receiving code discards all spaces in command line.
you would need to define one function for each command in the language and add the mapping into the keys[] array
The func_unknown() function is called if an unknown command is found.
In this code when the buffer is full, you get a warning message and I ignore that command line - basically submit it empty to the parser so that the main code would still know there was an incoming command but it was ill-formed. You could devise a different strategy like handling only what you got (but that comes with a risk if the last digit you got was 'L' and the command was really L2 or L' --> you'd execute something that was not intended.
30 years of bad experience with 'experienced' programmers getting c-strings wrong, shows that even with 'care' buffer overflows and missing nulls occur. That is why Microsoft banned c-string methods and replaced them with an alternate 'safe' library.
The OP asked for Arduino code, not C code.
The 'standard' in Arduino for text process in Strings.
Arduino Strings are much closer to what a matlab user would be accustom to and far easier and safer to use.
My suggestion is to start with Arduino Strings and see what the result is like. Only then to consider SafeStrings.
As Don Knuth said "We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil."