I2C: Optimization of the wire.requestFrom() function, or waste of time?

[EDIT: Experienced users on the forum are very skeptical about this idea due to it's complexity and believe it will be buggy, so I have abandoned it, in favor of using additional GPIO pins so that the slave can tell the master when it's got data]

TL;DR; I’m trying to send different sized packets of data from a slave to a master via I2C. I’m trying to make the transaction more efficient, by initially telling the master how many byte’s it’s going to receive, and then sending the actual data. I’m looking for feedback as to whether it’s a good solution.

I’m trying to achieve seamless 2-way communication via I2C, with 1 Arduino Master, and 2 Slaves, without the potentially problematic multi-mastering solution.

This is achieved by the master sending data to the slaves, and the master also requesting data from the slaves using the Wire.requestFrom() function.

The master sends data to the slaves whenever it needs to, and makes a request to the slaves every few seconds, to check if they have anything they want to send.

However, as others have noted, this wire.request.From() function is a bit weird. If you make a request of 32 bytes, but the slave only has 2 bytes to wants to send, the receive buffer will fill up with those 2 bytes, and then 30 more bytes of junk. There doesn’t seem to be an easy software fix to this, as it’s inherent to the hardware. More info here; Wire.h: Wrong number of bytes returned from the slave device (I2C) · Issue #384 · arduino/ArduinoCore-avr · GitHub

You can easily parse out a smaller variables from the rest of these useless bytes with an end marker, at the end of the packet you’re sending, as shown in my examples (using the wire.readbytesuntil function).

However, lets say I wanted to send various sized packets of data to the master every few seconds, lets say between 4 and 32 bytes. Since the master has no idea what is being sent, it will have the number of bytes requested constantly set to 32 bytes.

If I make a request for data every few seconds, and most of the time the slave is only sending 4 bytes, I will still be filling up the Master’s receive buffer with the number of bytes of the largest package, i.e. 32 bytes, every few seconds.

So, my question is if it’s inefficient for the I2C receive buffer to continually be filled up with bytes the size of the maximum packet size every few seconds, even if most of it is useless data?

To combat this, I’ve devised a simple solution.

NOTE: I am using character arrays in this example, for easy of debugging, but my final intended solution will involve the sending of byte arrays, and parsing out mostly numerical data.

Every few seconds, the master requests just 3 bytes of data. When the slave has nothing to send, it just sends back “0” and the null terminator.

When the slave does have something to send, the slave first calculates the size of what it being sent, and then sends this size to the master. The slave then immediately puts the array it intends to send into the Wire.Onrequest interrupt routine.

When the master receives this size, it then changes the number of bytes it’s requesting, to the size of the variable that the slave wants to send. (i.e. Wire.requestFrom(address, numbytes))

Then the master immediately makes another request and receives the actual data the slave wants to send. After that, the master goes back to requesting just 3 bytes until there’s another thing to be sent.

I have also tried to make the interrupt routine on the slave end as short as possible but let me know if it’s too long. I'm assuming it's fine since it just contains the changing of booleans and if-else statements, without any big calculations. The code seems to work flawlessly so far.

You can test this yourself by opening the Serial. Monitor on the slave end and entering in any data between a start and end marker “<” and “>”. E.g. type in < hello > and press enter. You can then see it being printed out on the Master end, along with how much is being placed into the receive buffer.

Let me know if this is a good solution, or if the continual filling up of the 32-byte buffer every few seconds is not actually going to produce that much of a performance disadvantage. And whether my solution could add too much complexity and potentially cause bugs.

Finally note that I’m already aware of UART and SPI and how easy it is to send the data bidirectional with these methods, but for interconnecting 3 or more microcontrollers, I2C seems to be the easiest solution.

Thanks

A quick summary of the programs;

  • The master requests 3 bytes every second from the slave.
  • When the slave has no data to send, it just sends back “0”.
  • When the slave does have something to send, the slave initially sends the length (in bytes) of the data.
  • Upon receiving this length, the master makes another request for the data itself.
  • The master then parses this data into the appropriate variables.
  • The master then dumps this received array, and then starts making new 3 byte requests for new data.
  • In this demo, you give the slave new data from the serial monitor.
    • The slave then sends the length of this data in bytes to the master.
    • Then the data itself is sent.

Code for the master

#include <Wire.h>

char receivedArr[30];


const byte otherAddress = 9;

byte numBytes = 3; //default
byte receivedLength;

void setup() {
  // put your setup code here, to run once:
  Wire.begin();
  Serial.begin(9600);
}

void loop() {


byte stop = true;

//STEP 1; 1 request made every second. The default number of bytes being requested is 3.

Wire.requestFrom(otherAddress, numBytes, stop);
Wire.readBytesUntil( '\0',(byte*) &receivedArr, numBytes);

//STEP 2; receiving the string length.

if(receivedArr[0] != '0'){ //this statement is true whenever there's new data, other than "0", being sent
                            //the first thing that will be sent from slave is the size of the data that it wants to send
                            
  byte receivedLength = atoi(receivedArr); //convert the received length from char to a byte
  numBytes = receivedLength;  //changes the requested number of bytes in the Wire.request function to the size of the incoming variable
  
  Serial.print("length = ");
  Serial.println(receivedLength); 

//STEP 3; receiving the string itself.

  Wire.requestFrom(otherAddress, numBytes, stop); //numBytes is now equal to size of incoming variable
  Serial.print("bytes in buffer = ");
  Serial.println(Wire.available());
  Wire.readBytesUntil( '\0',(byte*) &receivedArr, numBytes); //the actual variable we want to send comes through here. 

  Serial.print("received string = ");
  Serial.println(receivedArr); //print the variable
  numBytes = 2;
  

  memset(receivedArr,0,sizeof(receivedArr));//reset array  
}
else{ //only use else for Serialprint. delete after
Serial.println(receivedArr);  
}


delay(1000);
}

Code for the slave

#include <Wire.h>


const byte thisAddress = 9;


char sendNothing[2];

//editing array though serial line
const byte numChars = 30;
char receivedChars[numChars];

boolean newDataToSend = false;
boolean sendStringLengthToMaster = false;



boolean newData = false;

//bytes written to master
byte stringLength;
char stringLengthArr[3];
byte sizeofStringLengthArr;
 
void setup() {
  // put your setup code here, to run once:
  Wire.begin(thisAddress);
  Wire.onRequest(sendEvent);
  Serial.begin(9600);


receivedChars[0] = '0';
receivedChars[1] = '\0';
stringLength = 2; 

sendNothing[0] = '0';
sendNothing[1] = '\0';
}

void loop() {
    
    recvWithStartEndMarkers();
    showNewData();

}
void sendEvent() //interrupt routine that sends variable to master
{
if(newDataToSend == false){  
Wire.write((byte*) &sendNothing, 3 ); //STEP 1; by default it sends just "0" and terminator.
}
else{
  if(sendStringLengthToMaster == false){
  Wire.write((byte*) &receivedChars, stringLength); //STEP 3; lastly, we send the string itself
  newDataToSend = false;
  }  
  else{
    Wire.write((byte*) &stringLengthArr, sizeofStringLengthArr); //STEP 2; when message ready to send,
                                                                   //it first sends the length of the string
    sendStringLengthToMaster = false;
  }
}
}

//RECEIVE A NEW STRING THROUGH SERIAL LINE. 
void recvWithStartEndMarkers() {
    static boolean recvInProgress = false;
    //receivedChars[0]= '0';
    static byte ndx = 0;
    char startMarker = '<';
    char endMarker = '>';
    char rc;
 
    while (Serial.available() > 0 && newData == false) {
        rc = Serial.read();

        if (recvInProgress == true) {
            if (rc != endMarker) {
                receivedChars[ndx] = rc;
                ndx++;
                if (ndx >= numChars) {
                    ndx = numChars - 1;
                }
            }
            else {
                receivedChars[ndx] = '\0'; // terminate the string
                recvInProgress = false;
                stringLength = ndx;
                ndx = 0;
                newData = true;
            }
        }

        else if (rc == startMarker) {
            recvInProgress = true;
        }
    }
}

void showNewData() {
    if (newData == true) {
        Serial.print("length in bytes to send ... ");        
        Serial.println(stringLength);
        Serial.print("string to send ... "); 
        Serial.println(receivedChars);
        
        memset(stringLengthArr,0,sizeof(stringLengthArr)); //reset stringlength

        itoa(stringLength, stringLengthArr, 10); //converts to char array

        sendStringLengthToMaster = true;
        
        sizeofStringLengthArr = sizeof(stringLengthArr);
        
        newData = false;
        newDataToSend = true; //statement made after byte is parsed
    }
}

Example serial monitors. Com3 is slave, Com6 is master. "0" is received and printed when nothing else is being sent

If Master makes a request to the Slave to put 6-byte data on the Wire Buffer for onward transmission, the Slave will always put 6-byte data. This is the protocol between them.

byte m = Wire.requestFrom(sAddress, 6);
//m is always equal to 6 even the Salve sends less than 6-byte data.

void sendEvent()
{
     Wire.write(myArray, sizeof(myArray));  //sizeof(myArraty) = 6
}

You are discussing I2C Protocol; why is here UART Protocol?

The UART is irrelevant, it is just used to send data to the slave (via the serial monitor), which is then sent to the Master via I2C. I am demonstrating what I believe might be a more efficient method of sending different sized data via I2C requests.

However, I think I've explained everything a bit confusingly, so I'll try to make a more simple step by step process of what I'm trying to achieve.

Let the master request the number of data bytes first (1 byte binary). If the slave has bytes to send then start a second request for exactly that number of bytes.

Yes. That's essentially what I'm doing, but I'm using a NULL terminated character array (3 bytes max) for debugging purposes.

The intended solution is for a 1 byte request to be made every second for 1 byte binary byte variable. e.g. if slave has 20 bytes to send, it sends 20 in byte form.

However, you seem to suggest doing this in a 3 step process, which I'm not sure is needed. Step 1 would be to make a request for 1 byte. If the byte is above 0, then the master knows that the slave wants to send something, as well as the amount of bytes that it wants to send. Then the next request is immediately for the data itself, in the form of an array of bytes, which is then parsed afterwards.

Which 3 steps do you mean? I only can see 2 steps in my suggestion.

Sorry, seems like I misinterpreted you on this part.

Let me try to write it down in a different way, it is basicly the same as DrDiettrich and GigaNerdTheReckoning are saying.

Master : How much data do you have ? (1 byte request).
Slave : Nothing (single byte is zero).
Master : How much data do you have ? (1 byte request).
Slave : Nothing (single byte is zero).
Master : How much data do you have ? (1 byte request).
Slave : 5 bytes !
Master : Give me data (5 byte request).

This will work without confirmation and without better handshake. The Master always reads the 5 bytes after the Slave tells that there are 5 bytes.

It is possible to increase the safety by making two different requests, one for how many bytes and one for the data itself. I suggest not to do that to keep the code small. There are so many things that can go wrong (during startup, when there are more than 32 bytes, and so on).

There is no fix at all, because it is how the I2C bus works. You can put an extra software layer on top of it, and that is what you just did :smiley:

I don't think so. The I2C bus is best for fixed size packages, with sensors and EEPROM and so on. It was never intended to be a stream of variable length readable ASCII data.
The more complex your code, the more trouble you might encounter.

There are some serious troubles with your code.
The ReadBytesUntil does not include the zero-terminator. That is a big bug.
You have to decide if you are sending the zero-terminator over I2C or not. Then you have to explain with comments everywhere when the zero-terminator is added or stripped from the data.

Thanks for the feedback. I have found a lot of useful information from you across the forum and github about how to use and not use the I2C bus. I had lots of crashes trying to implement a multi-master system and you guided me in the right direction towards using requests instead.

Thanks also for simple explanation.

The ASCII data is only used for demo purposes here. My final application will be variable length binary data, where it is placed in a volatile byte buffer, and then parsed into numerical data.

I was mainly referring to the 2-way communication between 3 or more micro-controllers. If it's just 2 micro-controllers, then UART or SPI seems like a better choice.

Are you saying that sending a char array over I2C, along with the zero-terminator could be problematic for parsing the data?

...is C specific and should be removed from transmissions. Text transmissions better end with '\n' and binary transmissions should come with a byte count.

Interesting...what exactly do you mean by including a byte count in a binary transmission?

At the begin of a transmission the number of following bytes (payload) is transmitted.

1 Like

You should first clearly mention with solid example what problem you are facing to exchnage data between two controllers using I2C Bus. Instead, you have gathered together a lot of confusing information.

In I2C Bus, the Slave is a hardware programmed controller/sensor. It has been pre-programmed having pre-negotiation with the Master (the Protocol) how many data bytes it would be sending being requested by Master. There is no option for the Slave to decide whether to send data or not unless you implememt that requirement with additional software codes.

As I have stated that I2C is a Protocol; where, there is no handshking (which you want to implement) except the inherent ACK.

How to inplement the above dialogue?

When the Master executes the requestFrom() command, the Slave immediately eneters into sendEvent() routine and puts the pre-programmed amount of data bytes into the Wire Buffer. That data are automatically transferred to the Master over the I2C Bus and get stored into Matser's Wire Buffer.

When the above are coded, we get:

Wire.requestFrom(sAddr, 1);

Do you think that the Slave has access to the second argument of requestFrom(arg1, arg2) command when the sendEvent() routine has no option for formal arguments? Does the value of second argument really go to the Slave or it is used by the Master?

On further viewing, despite all my edits, my sketch is probably still going to confuse most people, so I'm sorry for being rude to you. I've actually found a lot of very useful examples involving I2C and UART communication that you've provided to the forum, so keep up the great work!!

1 Like

Your sketch isn't confusing so much, but your inital aproach. Concentrate on the desired result.

Follow the suggestion of post #9 from Koepel. I see several advantages:

  • it is just the usage of existing functionality of the library
  • your code will be portable like any other sketch relying on the Arduino i2c library API
  • you don't need to change anything in the i2c library which might lead that you break I2C protocol definitions

I appreciate your interest in adding users' hand shaking mechanism with the existing I2C protocol.

Please, keep your works going ahead.

Though, I2C supports ASCII transmission, yet users generally prefer binary transmission.