I'm transmitting data from one arduino to another over copper wire via the serial ports. I want to transmit the data as quickly as possible, but I also want to be sure that the data is coming through accurately.
The data is a series of coma separated fields (typically numeric values) combined together into records separated by carriage returns & line feeds.
What's the best way of checking that the records are being received correctly? Is a simple two-digit checksum like used on NMEA records the best options? or is there a better idea?
Why all the commas and line feed characters? If a human is going to read the data, that's important. But if another Arduino will read it, you can shorten the message by sending the data as a simple stream of bytes, not even converting numbers to decimal ascii. That would reduce the probability of an error in any single message, and speed up transmission. Alternatively you could reduce the baud rate while still maintaining the same rate of messages, which would also reduce the probability of errors.
Something else to think about would be Manchester Encoding, which not only allows error detection, but also allows for some error correction.
You're right, ultimately it's only going to be read by a machine, but in practice, during the development cycle it really helps if I can look at the data and visually confirm that it is or isn't working.
Something else to think about would be Manchester Encoding
Doesn't that require a common clock signal between transmitter and receiver?
Manchester coding does allow for error detection by itself, but not error correction unless its used with some other error correction scheme.
The downside of Manchester coding is that it halves the data thruput rate due to its overhead.
Anyway, my suggestion would be to get another wire and use SPI. That way you send a byte A, send byte B, and once B is sent, you got your byte A back. You mentioned in your other thread, that you only need single directional communication. This would fully utilize the spi bus and wouldn't require any computation on the slave device.
If you want to keep it somewhat human readable, the NMEA type checksum is the best way to go. It can fail to detect an error if you get two errors in the same bit position. If your transmission channel has a lot of errors and you cannot tolerate a glitch, then it might catch you out.
Another method is to send duplicates - send the same message 3 times and pick the best 2 out of 3 for every bit position.
The 16-bit CRC used by MODBUS is the next level of complexity. A CRC (Cyclic Redundancy Check) can be built with an arbitrary level of complexity, to detect and even correct errors. The maths behind it is quite advanced though.
All of those are one-way methods. If you can have the receiver send back an ACK or NACK for every packet then the transmitter will know which ones it has to send again.
Thanks for everyone's input. For the moment it looks that a NMEA checksum is the way to go. I'll investigate the other options if NMEA doesn't work out well.
Fulliautomatix:
Thanks for everyone's input. For the moment it looks that a NMEA checksum is the way to go. I'll investigate the other options if NMEA doesn't work out well.
Cheers
B... but! Using spi with the MISO for checking would be super simple, it wouldn't have any computational overhead (or extremely little) and you'd just have to use 1 more wire...
Plus it'd be almost bulletproof. Actually, it'd probably be as good as you can get in terms of error detection, but I don't feel like constructing a proof for that!
Hmmmmmmmmmmmmmmmmmm... It's such an ingenious idea, that I kinda have no idea why you don't wanna go that way, but... it's your project...
I don't know that the SPI echo method is totally bulletproof. Let's say for an instant there's an electric field that causes the slave end of MOSI to go high when the master holds it low. The slave then transmits high on MISO. But the same interference field applies to that, in reverse. MISO could be low on the master end.
But those two transmissions don't occur at the same time do they? So any interference would have to be synchronised to the SPI clock, which is extraordinarily unlikely.
So yes, in all cases including broken wiring, the SPI echo will detect all errors with an extremely high reliability.
CRCs can be useful in the face of heavy interference, when errors can be detected and repaired even if no packet ever survives without damage. A simple retransmission protocol can't correct errors if every packet is damaged, even if the damage is in a different place in each one. (But then you just make the packets smaller until there is a sufficient probability that a packet will not be damaged.)
Hi goodHen - you argue a good case, i will looking into SPI - the simple reason for my current plan to use NMEA style checksums is that I already know how to generate and check them! - so little new work for me!