Hi guys! I am trying to implement a gcode sender using arduino uno, sd card and display, to send to GRBL 1.1f in arduino nano,but I had issues with the stream method of waiting for ok , the machine stops in the middle of the work or keep always the laser on! so I figured out that is all about the streaming method but for some reason, I cannot find any working sketch using character counting! Maybe no one is interested, who knows.
So I need to always watch the buffer size to keep the characters sent below 127, and then subtract the number of characters corresponding to the line executed with each Grbl response.
The problem is that I do not know how to do that part. right now my code open the file, and send to GRBL lines of code and stops before the character counting exceed 127 so any idea about how to complete my code is always welcome. Basically I need to write an equivalent to this part of the phyton script:
grbl_out += out_temp;
g_count += 1 # Iterate g-code counter
grbl_out += str(g_count); # Add line finished indicator
del c_line[0] # Delete the block character count corresponding to the last 'ok'
if verbose: print "SND: " + str(l_count) + " : " + l_block,
s.write(l_block + '\n') # Send g-code block to grbl
if verbose : print "BUF:",str(sum(c_line)),"REC:",grbl_out
and this is my current work:
void sendGcode() {
//READING GCODE FILE AND SEND ON SERIAL PORT TO GRBL
Serial.print("\r\n\r\n"); //Wake up grbl
//delay(2);
//Serial.flush();
//while(Serial.available())
// Serial.read();
//----------------------------
if (myFile) {
unsigned int totalCountLines = 0;
unsigned int totalCountChar = 0;
unsigned int lastStringLength = 0;
sum = 0;
while (myFile.available()) {
String l_line = "";//create an empty string
l_line = myFile.readStringUntil('\n'); //it looks for end of the line in my file
unsigned int lastStringLength = l_line.length(); // previous length of the String//get the length here
unsigned int totalCountChar = lastStringLength;
if (sum > 127) {
//I must to continue here but I dunno how to count the ok´s received and then evaluate if it is safe to send another line! so this while loop is just for debug.
while (1) {
get_ok = Serial.read();
if (get_ok == 'o') {
get_ok = ' ';
Serial.print(sum);
break;
}
}
} else {
Buffer[i] = totalCountChar; //asign to the array
sum += Buffer[i]; // sum all the elements in the array
//Serial.println(sum); //sum is equivalent to g_count
Serial.println(l_line);//Yes you can send this line
i++;
//Serial.print(sum);
}
}
It is not a good idea to use the String (capital S) class on an Arduino as it can cause memory corruption in the small memory on an Arduino. Just use cstrings - char arrays terminated with 0.
And with the sort of program you are using there will be a lot of String processing
I confess I don't understand what you are trying to do. Do you mean that you want to read 127 bytes from the SD Card and send them to the the other Arduino after you receive an OK?
I want to send chunks that never exceed 127 characters, and wait until I get 'ok' from GRBL ,so let say I have 3 lines of 50 characters each in the file, I need to send only line 1 and 2 (100 characters<127) and manage to figure out when the other arduino(GRBL) is ready to accept the next line, this is when line 1 is processed and the buffer(GRBL) of has only 50 characters in it(line 2)
GRBL responds always with 'ok' when a line is processed.
I am stuck because I do not have skills in programing arduino, just a basic level, I did not know about cstrings for instance, so I will update the code thanks to you.
This is what they say about this streaming protocol:
...this protocol simply uses Grbl's responses as a way to reliably track how much room there is in Grbl's serial receive buffer. An example of this protocol is outlined in the stream.py streaming script in our repo.
The main difference between this protocol and the others is the host PC needs to maintain a standing count of how many characters it has sent to Grbl and then subtract the number of characters corresponding to the line executed with each Grbl response. Suppose there is a short G-code program that has 5 lines with 25, 40, 31, 58, and 20 characters (counting the line feed and carriage return characters too). We know Grbl has a 127 character serial receive buffer, and the host PC can send up to 127 characters without overflowing the buffer. If we let the host PC send as many complete lines as we can without over flowing Grbl's serial receive buffer, the first three lines of 25, 40, and 31 characters can be sent for a total of 96 characters. When Grbl responds, we know the first line has been processed and is no longer in the serial read buffer. As it stands, the serial read buffer now has the 40 and 31 character lines in it for a total of 71 characters. The host PC needs to then determine if it's safe to send the next line without overflowing the buffer. With the next line at 58 characters and the serial buffer at 71 for a total of 129 characters, the host PC will need to wait until more room has cleared from the serial buffer. When the next Grbl response comes in, the second line has been processed and only the third 31 character line remains in the serial buffer. At this point, it's safe to send the remaining last two 58 and 20 character lines of the g-code program for a total of 109.
While seemingly complicated, this character-counting streaming protocol is extremely effective in practice. It always ensures Grbl's serial read buffer is filled, while never overflowing it. It maximizes Grbl's performance by keeping the look-ahead planner buffer full by better utilizing the bi-directional data flow of the serial port, and it's fairly simple to implement as our stream.py script illustrates. We have stress-tested this character-counting protocol to extremes and it has not yet failed. Seemingly, only the speed of the serial connection is the limit.
attiny85nightmare:
I want to send chunks that never exceed 127 characters, and wait until I get 'ok' from GRBL ,so let say I have 3 lines of 50 characters each in the file, I need to send only line 1 and 2 (100 characters<127) and manage to figure out when the other arduino(GRBL) is ready to accept the next line, this is when line 1 is processed and the buffer(GRBL) of has only 50 characters in it(line 2)
If I understand things you want to always be in a position where GRBL has data it can work on. So when it says "OK I have just finished a line" it should already have the data for the next line.
It seems to me a simple way to achieve that is, at the very start send two lines (its buffer will be empty and it will have plenty of space) and after that every time you get an OK send the next single line.
If that won't work please explain what I am misunderstanding.
It seems to me a simple way to achieve that is, at the very start send two lines (its buffer will be empty and it will have plenty of space) and after that every time you get an OK send the next single line.
If that won't work please explain what I am misunderstanding.
Exactly! so the GRBL buffer has always data to work, because after this buffer there is a second one ,the planner of movements, so if we can keep the first buffer full or almost full, everything goes smoothy, otherwise, bad things happen
We can send any number of lines at any time, as long as the sum of characters in the buffer don´t exceed of 127 so I did that part in my code, at this point I can count the total characters and I keep track of the number of lines sent, I even stop when I cannot send the next line, but I am mentally blocked to manage the next part, count the 'ok' responses and determine which line was already processed ,then calculate how much room is in the buffer once the first line is done, then evaluate if I can send the next line, if not, wait for the second ok and so on.
I think this sketch would be very usefull for the CNC community since we could simple get rid of the Host Pc and attach another arduino, but all the projects I have found has the sower 'wait for ok' method and is not that reliable if you are making small movements or sharpy curves for instance.
I did my homework with cstrings but I am now more confused :o I need to chance :
String l_line = "";//create an empty string
l_line = myFile.readStringUntil('\n'); //it looks for end of the line in my file
but I do not how to do it since readStringUntil() reads directly from serial as default for the String class so I am struggling to change it.
it could work as we are not using the USB to serial chip but this is what they say about XON XOFF that is why i jumped right to the recommended method:
While sound in logic, software flow control has a number of problems. The timing between Grbl and the host PC is almost never perfectly in sync, in large part due to the USB protocol and the USB-serial converter chips on every Arduino. This poses a big problem when sending and receiving these special flow-control characters. When Grbl's serial receive buffer is low, the time between when it sends the ready-to-receive character and when the host PC sends more data all depends everything in between. If the host PC is busy or the Arduino USB-serial converter is not sending the character on time, this lag can cause Grbl to wait for more serial data to come in before parsing and executing the next line of G-code. Even worse though, if the serial receive buffer is nearing full and the stop-receive character is sent, the host PC may not receive the signal in time to stop the data transfer and over-flow Grbl's serial buffer. This is bad and will corrupt the data stream.
I will keep trying this night and will be back asap!
XON/XOFF was a form of serial handshaking I recall from the 80's, you send ASCII chars for pause and resume transmission which can happen mid-message. I've used this before on 8-bit machines, then PCXT's and some 486-586 machines.
The PC serial task is sending text? Characters or frames? You do know that serial has no guarantee but through use of parity bit(s) which default Arduino Serial has none and whether the read byte fits an expected pattern like only alphas and digits you can be pretty sure you have good data per byte so .. the PC can be sent an ACK per good char received or a NACK to trigger a resend, an XOFF instead of ACK and XON or ACK later to get next when ready and then you get solid data for your CNC, the PC only sends a byte if there's a go-ahead.
The PC can fill the controller buffer in nothing flat while the CNC is busy just moving. The PC can wait for handshaking and still fill the buffer faster than it can empty 1000x over. Is having solid data worth the not-really-a performance hit?
XON/XOFF flow control proved to be problematic and did not work at all on boards using the Atmel 8U2 and 16U2 USB converter chips. As such XON/XOFF flow control cannot be used on UNO or Mega 2560 platforms using these chips. For this reason the XON/XOFF flow control was removed from the code.
I thought it still could work as I am not using any usbto serial chip at all,only rx/tx of both arduinos but I did find that the main drawback in terms of use is that they were unable to send run-time command characters as feedhold, Reset and Status Report.
The PC serial task is sending text? Characters or frames?
This is a typical content of my gcode files,this one is for a small circle:
M05 S0
G90
G21
G1 F3000
G1 X10.15 Y5.2816
G4 P0
M03 S1100
G4 P0
G1 F400
G2 X8.6907 Y1.7586 I-4.9822 J0.0
G2 X5.1678 Y0.2994 I-3.523 J3.523
G2 X1.6448 Y1.7586 I0. J4.9822
G2 X0.1855 Y5.2816 I3.523 J3.523
G2 X1.6448 Y8.8046 I4.9822 J-0.
G2 X5.1678 Y10.2639 I3.523 J-3.523
G2 X8.6907 Y8.8046 I0. J-4.9822
G2 X10.15 Y5.2816 I-3.523 J-3.523
G1 X10.15 Y5.2816
G4 P0
M05 S0
G1 F3000
G1 X0 Y0
the grbl machine can use .gcode, .txt .nc so all of they are plain text.
I have not access to the machine today but I will stick with the character counting protocol since this is the method most GUI developers use on platforms like windows or linux.I got some ideas and more coffe so I will bring some results next post!
attiny85nightmare:
but I am mentally blocked to manage the next part, count the 'ok' responses and determine which line was already processed ,then calculate how much room is in the buffer once the first line is done, then evaluate if I can send the next line, if not, wait for the second ok and so on.
Doesn't the system I suggested in Reply #4 solve that by simplifying things?
XON/XOFF flow control proved to be problematic and did not work at all on boards using the Atmel 8U2 and 16U2 USB converter chips.
Sounds like a mainly software Linux person wrote that. In the total mess that is the Linux world things often just “do not work”, there is always a true explanation but it is often too complex for most people to find.
In the much more orderd and consistent embedded world a statement like that just points up the ignorance of the writer.
Yes Xon/Xoff will cause a bit of an over run but it is simply a matter of getting the buffers and thresholds the right size.
but I did find that the main drawback in terms of use is that they were unable to send run-time command characters as feedhold, Reset and Status Report
Sounds like you didn’t implement it correctly to allow these features to implement in a timely fashion.
Doesn't the system I suggested in Reply #4 solve that by simplifying things?
It seems to me a simple way to achieve that is, at the very start send two lines (its buffer will be empty and it will have plenty of space) and after that every time you get an OK send the next single line
Robin2 I can send the next line only if I know how room is available and I know that the line is under that number( /n included). so it will work in the file I used as example but not in all the files, unless I evaluate the available space and the length of the next line first. Lets say I have 8 lines of 5 characters each, so I can send all of them the first time with no problem (40 characters < 127) (available room of 87) but if the next line (9) have 90 characters I need to wait for the first ok until the first line is processed and the new room is 92.
Sounds like you didn't implement it correctly to allow these features to implement in a timely fashion.
Grumpy_Mike I did not implement GRBL, those comments where taken from the official webpage so as far as I know, GRBL 1.1 does not support XON OFF anymore, of course they where talking about using the USB/serial chip on board. I only implemented the humble code at the top of the topic(not the phyton script!), I have a very basic knowledge of the arduino environment.
So after reading all your help I think I have a mental image of the next step I will try again tomorrow and we will see.
attiny85nightmare:
Robin2 I can send the next line only if I know how room is available and I know that the line is under that number( /n included). so it will work in the file I used as example but not in all the files, unless I evaluate the available space and the length of the next line first. Lets say I have 8 lines of 5 characters each, so I can send all of them the first time with no problem (40 characters < 127) (available room of 87) but if the next line (9) have 90 characters I need to wait for the first ok until the first line is processed and the new room is 92.
Let's have some facts.
What is the longest possible line? And can you post an example of it?
if you really need to keep track of the bytes in the GRBL buffer you could maintain a circular buffer in the Arduino with the lengths of each line sent to GRBL. As you retrieve an OK you advance (or is it retract?) the front buffer pointer and when you send a new line to GRBL you advance the rear buffer pointer and save the length there. At any moment the total number of bytes in the GRBL buffer will be sum of the values between the rear and front buffer pointers.
However my preference would be a system that avoids that complexity.
I found the simple strategy of sending only the next line when I receive the OK prompt works fine, without any need to queue up the G code instructions. It does not cause a pause even in the most complex milling.
I wrote code in processing to do this, find it in my project here:- http://www.thebox.myzen.co.uk/Hardware/CNC_Conversion.html
After reading Grumpy_Mike´s project,I decided to try again the wait for ok method and after 4 hour I failed miserably >:( the machine keeps doing all kind of funny movements, I even changed the power supply with no results, this is the main code. At least all the jogging stuff is working, maybe you guys can take look and see what I cannnot. When I send code 950 I enter to the sd list and I can scroll down with another key,so I choose the file and use code 149 to send over serial but the machine stops before finishing the job and with the laser still on!
More coffee and good sleep...
#include <LiquidCrystal.h>
const int rs = 9, en = 8, d4 = 5, d5 = 4, d6 = 3, d7 = 2;
LiquidCrystal lcd(rs, en, d4, d5, d6, d7);
const long interval = 300;
unsigned long previousMillis = 0;
unsigned long currentMillis;
/////// remote control/////////
int start_bit = 2200; //Start bit threshold (Microseconds)Sony protocol
int bin_1 = 1000; //Binary 1 threshold (Microseconds)
int bin_0 = 400; //Binary 0 threshold (Microseconds)
// Connect a 38KHz remote control sensor to the pin below
int irPin = 6;
int key;
int b = 10; //numeric part of X and Y
// Tarjeta memoria Sd
#include <SD.h>
#include <SPI.h>
File root;
File myFile;
int p = 0; // it is a switch to turn on/off printDirectory()
char get_ok;
int g;
//variables ejes maquina
int x;
int y;
int WhichScreen = 1;
void setup() {
pinMode(irPin, INPUT);
Serial.begin(115200);
// SD card
pinMode(7, OUTPUT);
SD.begin(7);
root = SD.open("/");
g = 1;
//axis
x = 0;
y = 0;
// initialize the LCD
lcd.begin(16, 2);
lcd.clear();
lcd.setCursor(3, 0);
lcd.print("LASER NINJA 1");
lcd.setCursor(4, 1);
lcd.print("WELCOME");
delay(1000);
}
void loop() {
key = 0;
key = getIRKey();
currentMillis = millis();
if (currentMillis - previousMillis >= interval) { //avoid double click on remote
previousMillis = currentMillis;
switch (WhichScreen) {
case 1:
{
ejes();
if (key == 244) {
alante();
}
if (key == 245) {
atras();
}
if (key == 179) {
derecha();
}
if (key == 180) {
izquierda();
}
if (key == 224) {
homing();
}
if (key == 229) {
setzero();
}
if (key == 950) {
//Serial.print("going to sd card menu");
WhichScreen = 2;
} else { //Do nothing
}
}
break;
case 2:
{
printDirectory();// We have selected one file here
WhichScreen = 3;
}
break;
case 3:
{
if (key == 245) {
printDirectory();
// We have selected one file here
}
//--------------------
if (key == 149) { //send over serial good stuff!
Serial.println("G10 P0 L20 X0 Y0");
lcd.clear();
lcd.setCursor(4, 0);
lcd.print("WARNING");
lcd.setCursor(3, 1);
lcd.print("LASER ON!!");
sendGcode();
lcd.clear();
lcd.setCursor(3, 1);
lcd.print("DONE");
myFile.close(); //close current file
WhichScreen = 1;
}
if (key == 950) {
//Serial.print("going out of menu");
WhichScreen = 1;
} else { //Do nothing
}
}
break;
case 0:
{
}
break;
}
}
//delay(300);
}// end of loop
// SD card reading
void printDirectory() {//File dir,int numTabs
p = 0;
char* guarda; //store one single archive´s name at once
while (p < 1) {
File entry = root.openNextFile();
if (! entry) {
// no more files
root.rewindDirectory();//dir?
break;
}
guarda = entry.name();
myFile = SD.open(guarda);
lcd.clear();
lcd.print("Menu SD");
lcd.setCursor(0, 1);
lcd.print(guarda);
p++;
entry.close();
}
}
this is sendGcode():
void sendGcode(){
//READING GCODE FILE AND SEND ON SERIAL PORT TO GRBL
Serial.print("\r\n\r\n"); //Wake up grbl
//delay(2);
Serial.flush();
if (myFile) {
while (myFile.available()) {
while(1){
get_ok = Serial.read();
if (get_ok == 'o'){
get_ok = ' ';
break;
}
}
String l_line = "";//create an empty string
l_line=myFile.readStringUntil('\n');//it looks for end of the line in my file
Serial.println(l_line);//Yes you can send this line
}//do not delete hello!!!
}
}
Just looking at your sendGcode() function ... it should not be calling Serial.read() to check for an OK. That should be done elsewhere. The sendGCode() function should be something like this
void sendGcode() {
if (okToSendGcode == true) {
// code to send a line
okToSendGcode = false;
}
}
somewhere else in the program there will be a function that receives messages from GRBL and another function that is called when a new message is received and which checks for an OK. If it detects an OK then it sets the variable okToSendGcode = true; so that the function sendGcode() can do its thing.
Divide your program into small logical parts and it will be much simpler to figure out. For what I have been describing the code in loop() might be
Robin2; I followed your instructions and got this:
void loop() {
receiveWithEndMarker();
checkForOK();
sendGCode();
key = getIRKey();
if (key == 245) { //enter sd menu
key = 0;
p = 0;
printDirectory();//pick a file
}
if (key == 149) { //send over serial good stuff!
key = 0;
//okToSendGcode = true;//I need to manipulate this using the ok response from GRBL!
Start = 1;
}
void checkForOK(){
if (get_ok.substring(0)=="ok"){ //look for ok
okToSendGcode = true;
}
void sendGCode() {
if (Start == 1) {
if (myFile) {
lcd.clear();
lcd.print("LASER ON!!");
delay(100);
while (myFile.available()) {
if (okToSendGcode == true) {
//we get "ok" with jogging commands too not only when sending file.
String l_line = "";//create an empty string
l_line = myFile.readStringUntil('\n'); //it looks for end of the line in my file
Serial.println(l_line);//Yes you can send this line
okToSendGcode = false;// comment to send entire file.
}
}
lcd.clear();
lcd.setCursor(3, 1);
lcd.print("DONE");
myFile.close(); //close current file
}
Start = 0;
}
}
is not working,but when I comment both receiveWithEndMarker(); and checkForOK(); and uncomment
okToSendGcode = true I can send the entire file over serial when I pick a file and press button 149, so oviously is not detecting ok responses, looking further, if keep the code as is and set get_ok = "ok" and then I do get_ok.substring(0)=="ok" i can send the full file aswell, I did read the documentation of .substring(), it should be 1 but if i do that I cannot sent the file, if I do get_ok="this text ends with ok"
then substring does not detect the ok no matter I put 1 or 0.
working code yay!!! You guys should see the reaction here,everybody jumping and screaming! it really feels so good I am sure you know the feelings.
#include <LiquidCrystal.h>
const int rs = 9, en = 8, d4 = 5, d5 = 4, d6 = 3, d7 = 2;
LiquidCrystal lcd(rs, en, d4, d5, d6, d7);
/////// remote control/////////
int start_bit = 2200; //Start bit threshold (Microseconds)Sony protocol
int bin_1 = 1000; //Binary 1 threshold (Microseconds)
int bin_0 = 400; //Binary 0 threshold (Microseconds)
// Connect a 38KHz remote control sensor to the pin below
int irPin = 6;
bool okToSendGcode = false;
int p = 0;
int key;
int Start = 0;
#include <SD.h>
#include <SPI.h>
File root;
File myFile;
String get_ok;
String readString;
void setup() {
pinMode(irPin, INPUT);
Serial.begin(115200);
pinMode(7, OUTPUT);
SD.begin(7);
root = SD.open("/");
// initialize the LCD
lcd.begin(16, 2);
lcd.clear();
lcd.setCursor(3, 0);
lcd.print("LASER NINJA 1");
delay(1000);
}
void loop() {
key = getIRKey();
sendGCode();
if (key == 245) { //pick a file
key = 0;
p = 0;
printDirectory();
}
if (key == 149) { //send over serial good stuff!
key = 0;
Start = 1;
}
delay(300);
}
void printDirectory() {
char* fileName; //store one single archive´s name at once
while (p < 1) { //p=0
File entry = root.openNextFile();
if (! entry) {
// no more files
root.rewindDirectory();//dir?
break;
}
fileName = entry.name();
myFile = SD.open(fileName);//open picked file to send via serial while calling myFile!
lcd.clear();
lcd.print("Menu SD");
lcd.setCursor(0, 1);
lcd.print(fileName);
p++;
entry.close();
}
}
void sendGCode() {
if (Start == 1) {
if (myFile) {
lcd.clear();
lcd.print("LASER ON!!");
delay(100);
Serial.println("G10 P0 L20 X0 Y0 Z0");//no limits switches so good idea to wakeup GRBL by Zetzeroing.
while (myFile.available()) {
while (Serial.available()) {
delay(3);
char c = Serial.read();
readString += c;
}
if (readString.length() > 0) {
//Serial.println(readString);//echo
if (readString.indexOf("ok") >= 0)
{
okToSendGcode = true;
}
readString = "";
}
if (okToSendGcode == true) {
//we get "ok" with jogging commands too not only when sending file.
String l_line = "";//create an empty string
l_line = myFile.readStringUntil('\n'); //it looks for end of the line in my file
Serial.println(l_line);//Yes you can send this line
okToSendGcode = false;
}
}
lcd.clear();
lcd.setCursor(3, 1);
lcd.print("DONE");
myFile.close(); //close current file
}
Start = 0;
}
}
// CODE FOR SONY REMOTE ONLY
int getIRKey() {
int data[12];
int i;
while (pulseIn(irPin, LOW) < start_bit); //Wait for a start bit
for (i = 0 ; i < 11 ; i++)
data[i] = pulseIn(irPin, LOW); //Start measuring bits, I only want low pulses
for (i = 0 ; i < 11 ; i++) //Parse them
{
if (data[i] > bin_1) //is it a 1?
data[i] = 1;
else if (data[i] > bin_0) //is it a 0?
data[i] = 0;
else
return -1; //Flag the data as invalid; I don't know what it is! Return -1 on invalid data
}
int result = 0;
for (i = 0 ; i < 11 ; i++) //Convert data bits to integer
if (data[i] == 1) result |= (1 << i);
return result; //Return key number
}
I was a little worried about the delay(300) needed to avoid doubleclick of the remote control but it seems to be fine, I was like: if Grumpy_Mike said ok-response method worked fine and he made that impressive work with his DIY CNC and I am pretty sure he knows a lot more than me, I will keep trying this method.
I still need to add all the code for the jogging stuff that I removed.I think my main problem with coding for arduino is the fact that I do not have a way to know what is the best approach for certain problem. Like having a box full of tools but not knowing what tools are inside, do you know guys some book or resource to learn how to give structure to my projects or use a general approach to tackle a problem?
For example, do you think it would be a good idea to move all the current code to a state machine? so I could manage new features in an easy way or keep just the if{} then pattern?
Do you think I can shrink this part of the code?
