How to send massive data to a tiny serial buffer?

I have a plotter, which is supposed to draw Gcode scripts. I know very little about serial communication. Nothing I read about it says anything about a single issue, namely can the receiving unit halt the communication, if the buffer gets full? Is there anything in the protocol that makes the sender wait until the receiver has processed everything in the buffer?

I've been told that these kind of scripts can't just be hard coded into each sketch and it would be better to send the script through serial communication. So can I just dump the script as a text file to the COM port on my computer and rely on the system (Windows) that the data gets transferred? Or do I need to write my own software on Windows to perform the dump to prevent a buffer overflow in Arduino?

Is there anything in the protocol that makes the sender wait until the receiver has processed everything in the buffer?

Yes heir are quite a few ways of doing this. One is called Xon / Xoff, the Gcode receiver sends a special character Xon when it wants data to be sent and and another Xoff when the buffer is nearly full. This is a standard serial protocol.

You can also do the same thing with the serial handshake lines, that happens at the hardware level.

The other way I have done it is to send one Gcode at a time and have the Gcode device send an acknowledgement when that command has been processed. This is not a Gcode issue but one between the program sending the Gcode and the device receiving it.

I've been told that these kind of scripts can't just be hard coded into each sketc

I don't understand that, it sounds like rubbish to me.

Well, the Gcode scripts might get so large they exceed the size of what can be flashed.

The codes for CNC are not just "G" or go codes. There are "M" codes that define the speed of movement. So, your "G" instruction may take several seconds to complete. Then the next "G" code can begin execution. Being able to stop the transfer of "G" and "M" code lines is imperative for CNC to function.

Paul

A subset of the general flow control problem.

There are many solutions

Allan

What software are you using on your plotter?

What exactly do you mean by "tiny serial buffer" and where did you get the idea that it presents a problem?

Questions without context are impossible to answer.

While the developers may not have mentioned it I suspect it crossed their mind that GCode files are long.

...R

Robin2:
What software are you using on your plotter?

What exactly do you mean by "tiny serial buffer" and where did you get the idea that it presents a problem?

I'm about to write my own software for my plotter, which is an Uno, three servos and some wooden arms. It should read Gcode from the serial and perform accordingly. Right now it just draws beautiful artistic images of simple hardcoded geometric figures.

I read somewhere that the serial buffer is 64 bytes! A Gcode script file made by Inkscape can be 10 to 1000 kB. I want to dump such file to my plotter. By dumping I mean one action of say copy-pasting the whole script into a serial monitor in one turn. Or dragging the script file onto the serial monitor.

Obviously I have to code an extra level of buffer somewhere. Either on the PC side, where I regulate the speed of sending textlines according to the content of the textlines (what Paul said about M and G). Or on the Arduino, where I'm just supposed to read the buffer as fast as I can and translate it into a FIFO queue of very compact data (two floats for x and y and one byte bitfield of pen up, pen down and some other stuff).

Then have a look at the examples in Serial Input Basics - simple reliable ways to receive data. There is also a parse example.

My code empties the serial buffer faster than data arrives so the 64 byte size of the buffer is not a restriction.

But will you have enough space on your Arduino to store that much data? Maybe you should add an SD Card to your Arduino?

Or maybe you should write a PC program that only sends the GCode one line at a time when the Arduino requests it.

I have written my own code to control a small lathe but I interpret the GCode on my PC, not on my Arduino.

...R

Look at this project of mine Proxxon MF70 Miller CNC CONVERSION
I wrote a program in the Processing language to transfer the Gcode to the miller, this runs on the PC and transfers the code over to the Arduino controlled miller. You can get all the code from a link on that project page.

I want to dump such file to my plotter. By dumping I mean one action of say copy-pasting the whole script into a serial monitor in one turn. Or dragging the script file onto the serial monitor.

You could do this but you would need an SD card on the Arduino to store the file. Then at the computer end you would need a serial terminal program that transfers text files. Lots of free ones about depending on what platform you use. I think FileZilla will do this. https://filezilla-project.org/

Right now I don't have an SD card for the UNO. So I might end up with a system with some handshaking over the serial, which needs me to code something on the PC side, too. Must look into what you've done, Grumpy. Thanks for that.

I have also written some useful Gcode Apps for Processing. They were for an early version of Processing but it shows you what it can do.