Inter-arduino communication for displays

Well, I'll give you a little info on what I have..
I have 1 Mega 2560 doing engine controls, which of course involves reading lots of sensors, and turning outputs on and off.
It will have various tuning parameters, Some boolean, some integer, some byte, some floating point, and some integers in 2 dimensional arrays.

I have a 2nd Mega with a Sainsmart 3.2" TFT touchscreen I would like to display the parameters on, and be able to update the tuning parameters on the controller Arduino.

First off, How should I electrically connect them? I'd like to keep as many interrupt accessible pins on both units as I can... Apparently SPI doesn't support slave mode, so that's out..
How about just using pins 0 and 1.. I know they'd be shared with the USB controller, but would it be a problem? CAN bus would be an option (controller unit already has it) if the Touchscreen shield didn't kinda hog all the accessibility of the ports, but would require additional hardware. The two units will be about 6 feet away from each other, if that's important.

Secondly, What kind of data transfer method should I use that requires minimal parsing on the other end and doesn't require large amounts of bandwidth

So far I'm thinking I'm going to have to make a message with roughly the following structure

Message length in bytes (1 byte) with 1 reserved bit (to request the value rather than update it)
Message type (array, boolean, integer, string, etc) (1 byte)
Address (which variable we're modifying)(2 bytes?)
Data (could be up to about 250 bytes, but in practice will probably only be about 4 bytes)

For the Data, I think I'd limit floating point numbers to 10 bits for the base, 1 sign bit, 4 bits for an exponent, and the 16th bit would be the sign of the exponent..
It would give me 4 significant figures from 1e-15 to 1023e15 both positive and negative.. It should work

I'd really like to hear any criticism on this so I can avoid pitfalls before I spend massive amounts of time designing it!

Using one of the other serial ports on the Mega will probably be the easiest. For reliable communication you need to add some extra fields in your messages. A start byte that you can test against as well as a end byte. You can then know if a complete message was received (using those two in combination with the length indicator in the message); you might possibly want to add a checksume or CRC to the message as well to make sure that it's not corrupted.

The internal format is totally up to you.

Note:
With a message length indicator of one byte and using one bit to indicate request / set, you will only have a maximum size of 128 data bytes (so 250 is out of the question).

Yeah, As I was writing it I decided to put the request bit in there and just forgot to update the 250 byte limit (Or I was checking to see how alert you were… you passed)

If I have any sort of special character bytes, doesn’t that pretty much force me into keeping a high bit reserved for each byte transferred, so that it’s impossible to confuse a start byte with anything else… For example if I use 0xFF as a start byte, I can’t have it showing up anywhere else right?..

Good idea on the checksum/CRC… Even if I don’t implement it I may reserve the space for it… Perhaps immediately after the message length byte?

Are IO pins 0 and 1 the only ones you can use hardware serial on… is using software serial going to be a significant bottleneck?

Have you looked into wireless. I use NRF24s and my own message protocols. They have a built in CRC check and other clever stuff and should the need arise, ways to bi-directional comms, and you can have two doing the good work and a third just listening out and monitoring what is going on for development purposes (without having to impose on either of the working units). I hook up a ‘listener’ to the ESM.

Your environment (inside a car?) may make it impossible to get good TX/RX though depending on where you place them… but may alleviate hard wiring issues?

I will look into increasing the speed of my ESM, possibly later this week, to hit the baud rates you mention. Currently its top is 250000 - is that the version you have?

Cheers Alan

The mega has multiple hardware serial ports. I like that idea, just for the simplicity of it.

You may find some ideas in Serial Input Basics. The 3rd example will be most reliable. You could easily add a check-sum into it.

Design the receiving system first because it easier to get the sender to comply with the requirement.

It is probably easiest if the same set of data is sent every time even if some or all of it has not changed.

Sending human-readable data makes debugging very much easier even though it uses more bytes. I would only opt for non-human-readable data if I actually needed the extra performance due to smaller data quantities.

...R

Yes, it is the version I have... It's just a useful tool, and serial comms have improved in speed a lot.. I can hardly fathom why the defaults are nearly always 9600.

I have kinda thought about wireless.. bluetooth particularly because then I could just broadcast the message to a smartphone/computer to do the displaying.. but the drawback there it's not broadcast.
I think I'll keep wireless on the back burner for now, but include it in design consideration... Most likely, for simplicity, I'll just use a hardware serial port.. will read up on them.

Thanks for the link Robin2... it's 2 am and I'll study it when my eyes aren't buggy.

Since I am thinking of enabling the second arduino to request a data field, weighing the performance options of just sending all the data a couple times a second regardless of whether it's needed or not as opposed to the overhead of having to parse a request and send it back...

Rx7man:
If I have any sort of special character bytes, doesn't that pretty much force me into keeping a high bit reserved for each byte transferred, so that it's impossible to confuse a start byte with anything else... For example if I use 0xFF as a start byte, I can't have it showing up anywhere else right?..

No, the trick is to sync on the start-of-message and next read the number of bytes indicated by the length plus one; you know that the last byte must be an end-of-message byte. If it's not, you're out of sync. If it is, you can use the checksum/CRC to make sure.

Rx7man:
Good idea on the checksum/CRC.. Even if I don't implement it I may reserve the space for it... Perhaps immediately after the message length byte?

CS/CRC are usually the last bytes (after the actual data). A one-byte CS is very easy to implement; just sum the bytes that you are going to send (excluding start and end markers) into a new byte and subtract the calculated value from 0x100.
At the receiving end, just sum the full message (excluding start and end markers); the result must be 0.

Rx7man:
Are IO pins 0 and 1 the only ones you can use hardware serial on.. is using software serial going to be a significant bottleneck?

The Mega has multiple hardware serial ports; as I said, probably the easiest

Note: maybe add a protocol version in the message as well; in case you ever change from CS to CRC or whatever changes are needed, receivers can tell the sender that they don't understand the message.

You have quite a big block of data with NRF24 which is sent everytime - you can pack in a lot of useful stuff in one message.

You can do a kind of broadcast. More than one NRF can read the same message depending on how you do it.

OK, I'm starting to write the 'reader' part of it.. it'll be a long process.

If I have trouble with electrical interference on the serial port, I might look at RS485, which is has much better noise immunity, and the chips are probably pretty cheap... the only issue is I have to make a shield or some sort of adapter to use it (grumble)... time will tell.

Thanks for the tips so far.

How big is the serial buffer? If I'd limit my message length to 100 bytes or less, would the following work?

static char MessageStartCharacter = 0xFF;
byte MessageLength = 0;

   if (Serial.available() > 0){
      //perhaps start a timer here to set a timeout to get a complete message

      if (Serial.available() >= Serial.peek()){ //There are enough bytes in the buffer to satisfy the message length
        MessageLength = Serial.read(); //remove one byte from buffer, set message length

         if (Serial.peek() == MessageStartCharacter){

          //I have a complete message
         //parse message
        }
     }
   }

Rx7man:
OK, I'm starting to write the 'reader' part of it.. it'll be a long process.

If I have trouble with electrical interference on the serial port, I might look at RS485, which is has much better noise immunity, and the chips are probably pretty cheap... the only issue is I have to make a shield or some sort of adapter to use it (grumble)... time will tell.

Thanks for the tips so far.

Ready to use RS485 to TTL translation modules are available for cheap.

On an Uno, the internal buffer is 64 bytes; might be different on a Mega. As far as I know you can change it by modifying the library.

I suggest that you don't check length first. You need to sync on start-of-message. So the below might be a better protocol.

[SOM:1][datalength:1][version:1][command:1][data:n-2][cs:2][EOM:1]

For CS you can use a single byte, for CRC16 you need 2 bytes. Note that datalength usually does not include EOM.

After syncing on SOM, read datalength. Check that datalength does not exceed the length of the buffer where you will store your data that you read from the serial port; inform sender that something is wrong (optional) and cleanup the mess.

Next read all bytes as indicated by datalength and store in your own buffer. Read next byte; should be EOM. If not, inform sender that something is wrong (optional) and cleanup the mess.

Rx7man:
How big is the serial buffer?

You must not have looked at my links. My code allows you to receive as much data as you want as long as you have enough SRAM. There is also a parse example. Why re-invent the wheel?

If there is some aspect of my code that does not meet your need let me know. It can probably be adapted easily.

...R

Peter2, I did miss a link IN the link you posted to the example with a buffer... I would like to be able to use a variable length message at some point, I think your example does lend itself to that with a little extra work... Perhaps setting the buffer size to the largest reasonable value you expect, and then having it read the first byte as the actual message size and considering the message complete when it gets to that number of bytes.

You are right, I am reinventing the wheel, but I'm learning a lot while doing it.

Rx7man:
Peter2

? ? ? :slight_smile:

Because the Arduino has very limited SRAM it is best to pre-determine how it is used. I would certainly set aside space for the longest message. Once that is done the system in examples 2 and 3 in Serial Input Basics will receive any message of that length or shorter.

Managing memory usage on-the-fly can easily lead to corruption and hard-to-debug problems.

…R

Doh.. I mis-remembered your name.. I'm sorry

Yes, I have run into the problem of limited SRAM already on my Uno with serial.print getting all corrupted.. The Mega is a little more capable in that area thankfully.

I was thinking of expandability to other hardware, such as acboother's suggestion of NRF wireless, and am thinking that immediately after the Start marker I should perhaps reserve a 2 bytes for a destination address and sender address with the ability to broadcast to any address (MSB of destination address as a flag for a broadcast message?) For the size of my forseeable 'network' size I might be able to do it all in 1 byte... 1 bit for broadcast flag, 1 bit for something else, 3 bits each for sender and destination addresses.

[SOM:1][version:1][command:1][datalength][data:n][cs:2][EOM:1]
       |                                |
       +-------- header ----------------+

First read the header. Once you know the datalength, you can dynamically allocate memory using malloc (or calloc described on the same page) and read the data part in that buffer. Once you have processed the data, do not forget to free it using the free described in the malloc link!

  • read byte; ignore everything if it's not SOM
  • read header bytes (version, command, datalength)
  • allocate memory for buffer; don't forget to check if it succeeded
  • read data into buffer (read datalength bytes)
  • read CS/CRC (2 bytes; for CS you can limit to 1 byte)
  • read byte; if not EOM, free memory and ignore everything
  • verify cs / crc; if no match, free memory and ignore everything
  • process data
  • free memory

Note that it is absolutely necessary to free the memory at the moment that you no longer need it !! Forgetting to do so will cause a memory leak and you will end with hard-to-find bugs. A fixed size buffer (with a defined maximum size) is a lot safer from that perspective.

You can add address fields in the header for source and destination. Instead of using a bit to indicate broadcast, use a predefined address (e.g. 0x00 or 0xFF); it's more efficient If you limit the addresses to 4 bits (max 16 - 1 devices in the network), you can combine source and destination in a byte.

But I get the feeling that you're already implementing something that you have not completely defined yet. Think about what you want to achieve at the end, get it on paper and write code according to that.

Personally I’m not in favour of dynamically allocating memory with the alloc family on a micro processor.

There is no swap space on an Arduino so if at some time you need the buffer you might as well pre allocate it at the beginning. If you don’t know the max size then you have to design more sophisticated solutions for when you don’t have enough and there is already limited programming space…

Also you can save processor (and development/debugging) time and memory overhead by not having to use alloc…

Alan

sterretje:
Note that it is absolutely necessary to free the memory at the moment that you no longer need it !! Forgetting to do so will cause a memory leak and you will end with hard-to-find bugs.

That’s why I recommended a fixed size buffer in Reply #15 and did not refer to MALLOC.

And figuring out when you no longer need the memory may not be simple.

Also, you have to KNOW that there will be enough memory available for malloc() when a long message arrives. If you have freed the memory earlier (like 1 second earlier) so something else can use that space how do you ensure that the something else has finished with it?

Finally all this malloc-ing uses up CPU cycles that might be used for something more useful.

Different story entirely if you have 1.5GB of memory and 1.6GHz and an operating system to protect you from foolishness.

…R