I'm sending commands from Processing to the Arduino through Serial (via JSON handling), and responding from the Arduino to see them received/handled in the Processing Serial window.
The issue I'm having is if I send the packets through Serial too fast (too close together?) from processing, the Arduino locks up when the Serial Buffer is flooded. How can I reset the serial buffer to receive more packets, or should I be blocking incoming Serial traffic until the Arduino has properly handled the previous packet?
show us your Arduino code, please. I'll bet there are improvements to be made, but in the last, yes, you'll likely have to throttle your demands. That's a one-lung, 16 MHz cpu, with likely A PC of some form throwing stuff at it.
Sorry, gross assumption there, I'm talking about an Arduino Nano. You might have something different, how about telling us about it?.
I'm using an Arduino Minima. still single serial, but more processing power and memory.
The code sending strings over Serial to the Arduino is in Processing, which is also receiving serial message back from the Arduino to be able to debug operations.
I don't know the Minima, nor Processing, but you have three options, really.
Throttle Processing, or
monitor Serial.available() and process as fast as you can, or
throw away commands if you get too many. Basically, dump excessive input from processing, and handle a complete message.
If it was me, first thing I'd do is crank down the baud rate in Processing, because that slows things down "at source". You'd have to be doing a lot, even in an Uno, or do it very poorly, to not be able to handle incoming messages at 9600. If Processing doesn't give you that option, then you're back to fine-tuning your process, AND learning to throw away things in order to keep up. Does processing have any 'software handshake' options, like XON-XOFF, in wherever you setup your serial?
That's my current solution, but it's less than ideal. Processing is the GUI for the controls that represent the packets sent to the Arduino, I'd like it to be more responsive.
monitor Serial.available() and process as fast as you can, or
I'm trying to come up with a clean solution for this. I.e. The Arduino ignores any new serial traffic until it's completed processing the current one. But when it does will it intercept a partial serial message in progress?
Does processing have any 'software handshake' options, like XON-XOFF, in wherever you setup your serial?
I wish, no it doesn't I'm having to code the handshakes on my own. To me this is the best solution. Once the packet is handled the Arduino sends a message back to Processing, allowing it to send the next Packet. This is what I'm working on, but was hoping someone had a slick solution already for it.
It may be worthy to note I'm using Serial.readString() not Serial.read(). It seems everything I'm finding in searches are related to Serial.read(), very little in respect to Serial.readString().
Of course it can, its' your code. But the serial buffer isn't infinite. You have to look at it, when you're ready, and decide - hmm, buffer is nearly full, better discard a message, and on that basis strip characters until you hit an end-of-message marker(line feed, whatever), then process another command, OR, well, buffer isn't filling up, so process from start.
That's what I'd try, anyway. It all depends if those processing messages are all important.
Yup. Serial is sorta dumb that way. You don't get a little flag that waves, saying, Hey, I've got a new complete message for you. But many people have realized that, and simply written a buffer handler that pulls characters from serial and sets a flag when a complete message is in. It might need to get smarter, though, if it's possible to queue up more than one...
Lastly, try to write your code to keep up. That sounds kinda trite, but it's the fact of the matter. I still haven't seen any code; I assume this is all theoretical, "maybe tomorrow I'll write it like this" kinda stuff; so this is where I walk away. Good luck!
Oh, and yeah, String. Not so much. Might work fine on the Minima, ask others(isn't there a minima forum section?); it's there on the Uno, etc. but it's a memory thrasher, and the Uno doesn't do memory management, etc. so most avoid it like the plague. It might be a great tool, elsewhere, but it's deathly on an Uno.
I guess that's what I'm learning, I need to convert it to a faster/simpler Serial.read() with begin/end-of-message char. I'm reading it's a few ms between char reads vs hitting the timeout of 1000ms with Serial.readString(), which is WAY faster. This will also remove partial serial messages as I will only parse data that's between my begin/end chars.
A fact that should have been mentioned in your initial post.
Again drip feeding information does not help.
Yes Serial.readString() is useless for your problem. The big problem is that it only knows data has been received when it has not received anything for a second, which is useless for what you want.
So I would crank up the baud rate and send a header byte as the first byte of your message from processing saying how many bytes are in the message up to the size of your input buffer.
Then read that many bytes, and send an acknowledge byte back to processing. If that message byte is equal to the buffer size, then processing should hold off sending any more data until it receives the acknowledge byte. If it so happens that the total message is finished at this point, then it sends the next message with a zero number of bytes header, otherwise it continues until it sends you less than a buffers worth of data.
I thought it would be more of a generic question/answer, but the solution was far from what I was looking for. I see now my initial issue was with using readString() (so i should have opened with that). Changing to Serial.read() made a huge difference and even more so setting the read to the fixed size of the package sent.