When using timeouts in a state machine - make sure you exercise the timeout in your unit test...
I have a fairly simple serial communication between two arduinos. One sends a command, the other reads, performs the action and sends an ack.
Problem:
Unless I add delay(10) into the loop on the receiver it fails to read any of the packets.
Before I go an either wire up my scope or write some further debug tools, am I missing anything obvious due to hardware behaviour?
Without the delay in loop() on the receiver, it is basically repeatedly calling Serial.available(). One worry is whether this is interfering with the behaviour of Serial outside of the scope of the code?
The setup
2 arduinos (one nano one uno)
TTL to Rs485 interface boards with about 5m of cat5 between them.
Communication is at 115200, 7N1, 4bit nibbles sent encoded with 7,4 hamming code, the nibbles are decided by a fairly simple state machine into packets.
The state machine is well tested and copes fine with any valid buffer state, including partial packets (it will either complete the packet on a future receive call or time it out)
If I can’t track down an alternative possibility I’ll probably start the nano echoing back everything it receives at the serial level back to the uno (which has an lcd touchscreen and I have some debug utilities I can upload for this). However that’s fishing in the dark and I’d love to know a rational reason for this, and how it fix it without using delay, as that makes it impossible for it to keep up with traffic flow when I up the baud rate and have multiple devices on the line.
If possible, you should always post code directly in the forum thread as text using code tags:
Do an Auto Format (Tools > Auto Format in the Arduino IDE or Ctrl + B in the Arduino Web Editor) on your code. This will make it easier for you to spot bugs and make it easier for us to read.
In the Arduino IDE or Arduino Web Editor, click on the window that contains your sketch code.
Press "Ctrl + A". This will select all the text.
Press "Ctrl + C". This will copy the selected text to the clipboard.
In a forum reply here, click on the reply field.
Click the </> button on the forum toolbar. This will add the forum's code tags markup to your reply.
Press "Ctrl + V". This will paste the sketch between the code tags.
Move the cursor outside of the code tags before you add any additional text to your reply.
Repeat the above process if your sketch has multiple tabs.
This will make it easy for anyone to look at it, which will increase the likelihood of you getting help.
If the sketch is longer than the 9000 characters maximum allowed by the forum, then it's OK to add it as an attachment. After clicking the "Reply" button, you will see an "Attachments and other settings" link.
When your code requires a library that's not included with the Arduino IDE please post a link (using the chain links icon on the forum toolbar to make it clickable) to where you downloaded that library from or if you installed it using Library Manger (Sketch > Include Library > Manage Libraries in the Arduino IDE or Libraries > Library Manager in the Arduino Web Editor) then say so and state the full name of the library.
Have a look at the examples in Serial Input Basics - simple reliable ways to receive data. There is also a parse example to illustrate how to extract numbers from the received text.
The technique in the 3rd example will be the most reliable. It is what I use for Arduino to Arduino and Arduino to PC communication.
You can send data in a compatible format with code like this (or the equivalent in any other programming language)
A very common issue using serial is to assume erroneously that when serial.available tells you that something has arrived that the entire packet can now be read. e.g.
@pert
I’m using multiple libraries I’ve written and posting them is just going to create excessive confusion. I’m confident the libraries are correct as I have thorough test cases which pass. However will be extending the test cases into the error scenarios to be sure of that.
@Robin2
Thanks, I’m a little beyond that. I have a debugger that runs on the wire using an uno and an lcd display which understands the packets and the hamming code and can debug in place with the appropriate cable run.
@wildbill
As I said I’m running a state machine for the packet decoding, guard patterns on the available size before reading each part of the packet and test cases for partial packet handling. I will be extending these test cases for more paranoid examples as I don’t have 100% coverage on the bad path cases. Obviously the test harness is using a mocked serial class inheriting from stream which is basically a wrapper on a fifo - but it serves for automated testing
There should be no need for delay in your receiver code. That there apparently is, suggests to me that you have a harder to see version of the issue I described. So no magic bullet I'm afraid, you're going to have to put on your debugging trousers
Agreed. I only noticed the delay fixes it by chance in adding a serious of debug outputs that would wait a second (so as to output at a readable speed on the lcd)
I think some unit tests for receive against empty buffer is the way to go (and validate my dummy serial replicates the return -1 for no data... on read())
Though it’s different problem as I’m dealing with binary rather than text - which unfortunately makes the serial monitor link somewhat useless.
However I have just realised the obvious in writing this. Just run a serial test from the pc using python... will be an easier way to debug. Should have tried that sooner.
The idea being able to send any arbitrary length packet (capped at a arbitrary 36byte payload for now) of any 8bit bytes to a specific node on a rs485 link, cope with noise on the link with up to 1 in 7 bit flips, validate packet correctly received and not have any error cases that can cause a fail state in the state machine that can’t be solved with a timeout.
I’m thinking that @wildbill might be on the right track with somewhere that is checking for non zero bytes available and reading >1.... obviously shouldn’t be the case but should is such a wonderful term in coding... will expand tests to ensure that’s not the case then switch to python serial comms.
Shall we have a bet ?
pert, Robin2, wildbill and me can probably spot the problem in 5 minutes, no matter how long the code is. If the code is written in a bad way and it is a memory or hardware problem, then we might need to do some tests and that could take up to one hour.
But at least we can tell where to insert Serial.println() to keep track of the incoming data, or how to use memory more efficient, or to avoid the String in a Arduino Uno, and so on.
Taking it as confirmation that my original assumptions on the hardware (ie that this should work) were correct.
Won’t be able to get back to the code until Wednesday at the earliest but just from memory I’ve a few ideas as what is likely. Best guess a #define for an available length check is wrong and in the tight loop means I get a -1 for a read and then discard the packet as invalid.
Bit of unit test improvement should solve this, and python implementation of the protocol for pc debug will also help.
simm42:
Though it’s different problem as I’m dealing with binary rather than text - which unfortunately makes the serial monitor link somewhat useless.
Three comments ...
Although my examples are written to receive human readable text the concepts apply equally well to binary data. IIRC there is a binary example in the older version of Serial Input Basics
Unless the extra performance is essential sending data as human readable text makes debugging much easier.
It should be straightforward to get the Arduino to print out the byte-values of binary data for debugging purposes - for example to print 65 if it receives an 'A' (or 0x41).
I think a python based serial debugger is the way to go. A simple set of functions to send in the protocol I’ve implemented, read on the wire data, print that plus the decoded data in hex or decimal... fairly trivial really. Just hadn't thought about connecting a Python console to the arduino com port.
The lcd touchscreen debugger I have already prints a hex dump of the decoded packet. But it’s a slow debug process given the latency in interactions (though has the advantage of being at the deployment site rather than tied to a pc). Being able to dynamically change one side on the fly will definitely improve things. Integration style tests become possible also.
I’ve always disliked the serial monitor as inadequate - it’s basically debug print only which is very 1980’s - particularly given its text based nature.
Performance not strictly speaking an issue at this moment but scalability is a major concern (>200 slave devices on a shared bandwidth link with low user latency performance if an input signal is given on any of than) - and the performance is already impacted by the error correcting and overhead.
And the requirement for reliability which is solved by using hamming code negates any ability to send in printable ascii characters only. Apologies if hadn’t been clear on that.
simm42:
And the requirement for reliability which is solved by using hamming code
What is hamming code ?
but scalability is a major concern (>200 slave devices on a shared bandwidth link with low user latency performance
Without more details I can't make sense of that.
I am coming to the conclusion that you are trying to solve a problem that you have not clearly and completely explained. That makes it very hard to give useful advice.
Robin2:
What is hamming code ?
Without more details I can't make sense of that.
No point reinventing the wheel - its been done so well already.
Turns out that when you have a temporal component to your state machine (timeout) you need to exercise that aspect in your unit tests or you never see that behaviour exposed (Doh!). Once timeout hit it ended in a state where every further packet would timeout. Never hit it in the unit tests as they had no delays so never triggered a timeout.
On the plus side I now have a python serial interface I can run in the console that works at 7N1 and handles hamming code and the packetisation I've used (on the sending side), will write the deoder when I need it.
Are you doing the unit tests via the Python interface or do you have some sort of an Arduino unit testing application/library?
Tbh, although unit testing is a necessity in developing large scale software projects, but here in the world of embedded systems, it's much easier (and quicker) to use print statements. There are so many more edge cases and special considerations in embedded systems compared to normal software development - and writing unit tests for each of these is largely impractical, IMHO.
Even still, I'd be interested in looking at your final code - both Arduino and Python.