Random i2c errors, need help to stomp them out please!

The RTC is the only i2c item that gets power from the Arduino. All the LEDs on the ribbon cable are powered directly at the lights through separate power input. The voltage is not connected to the ribbon cable.

To confirm, you recommend a 0.1uF (100nF) cap between the Arduino and RTC?

Should I also cut the head off the Arduino barrel jack, connect a cap (what spec?) and run it into Vin and Gnd?

Can I still put 2 100uF/25V caps at the power supply? I worry about the motor causing noise in the signal.

The power supply I thought grounded both outputs automatically to each other? If not I thought the GND connection from the Arduino to the stepper motor control would connect the grounds.

Should I put a cable at the power supply to connect the grounds of both outputs?

Again, thanks for your attention to my problem : )

See the caps near the ICs here? One side connects to a devices +5V pin, the other to ground, right at each device. That is what we are talking about.

Get those added to start, I have to run, will address the rest later.

It's odd that it fails on the Wire.endTransmission. That shouldn't fail in the sense of going into a loop. Do you check the return code from endTransmission? If it is not zero, what do you do? A decoupling (0.1 uF) capacitor between +5V and GND right at the clock might help.

If you don't check the return code you may get bogus data back from the clock. What happens then?

Also, do you have the clock battery connected? It isn't shown on the circuit. I have read the clocks can fail without the battery.

I don't think it's odd that it's getting errors. I found many posts here of people with similar problems. It seems i2c was made for chip to chip communication and not for long cable runs : )

Apparently the line conditions must be perfect or else freezes can occur. It doesn't go into a loop, it just stops the Arduino and won't continue the rest of the code.

I never checked the return code for the wire transmission. I just have error checking code that makes the Arduino grab the time from the RTC again if un-clock like code is found. (time doesn't go backwards, or skip, etc.)

I've put some code in now to print the end transmission result. But many of my transmissions are LED instructions, not just RTC. I've seen the Arduino hang on i2c LED transmissions as well as RTC transmissions.

Yes, the clock battery has always been connected. I didn't put it in the diagram because it is part of the entire RTC breakout board.

Thank you for helping me : )

How do you know the problem is in the I2C endTransmission, and not somewhere else?

I'm curious. What is your project?

I take it that your switching speeds on the i2c cable are not very fast. Is that true?

How do you know the problem is in the I2C endTransmission, and not somewhere else?

I put a lot of Serial.print() lines in my code. Everything works before the Wire.endTransmission() line, but it never comes out of the Wire.endTransmission() line when there is a freeze. The Arduino is just stuck there.

I'm curious. What is your project?
I take it that your switching speeds on the i2c cable are not very fast. Is that true?

It's an artistic clock. I'm not sure what you mean by switching speeds. If you mean how often am I sending i2c communications, then it varies. But it is not a non stop stream of commands. I tried to put as many delays in between communications as possible. I could put more in. But for example it has frozen after a 30 second delay. I put the big delay in just to make sure everything was quiet when the transmission was sent =(

SparksAlot:
I'm not sure what you mean by switching speeds. If you mean how often am I sending i2c communications, then it varies. But it is not a non stop stream of commands. I tried to put as many delays in between communications as possible. I could put more in. But for example it has frozen after a 30 second delay. I put the big delay in just to make sure everything was quiet when the transmission was sent =(

I was just wondering if your data rate was slow enough to use a slower i2c clock. But I'm not sure what clock speed the Wire library uses, or how to change it. So that idea is probably not going to work.

It's not surprising that the problem comes up with the Wire.endTransmission command. The Wire.send commands just put the data in a buffer. The Wire.endTransmission command actually sends it. It does seem like your 20-foot cable is causing the occasional problem.

Perhaps it's time to incorporate an I2C extender into your design. http://ics.nxp.com/products/i2chubs/#Introduction . I'm guessing that if put a scope on the signals they will look like crap!

How about posting some code? The endTransmission shouldn't go into a loop for no reason. Looking at the code there are two places where it loops. Otherwise, no matter how bad the state of the lines, the code should exit. It might do bad things, but it should exit. These are the two places:

 // wait until twi is ready, become master transmitter
  while(TWI_READY != twi_state){
    continue;
  }

... and ...

  // wait for write operation to complete
  while(wait && (TWI_MTX == twi_state)){
    continue;
  }

Both of these are interrupt driven. That is, twi_state is a volatile variable that gets changed by an interrupt. So the first question is, are you fiddling with interrupts? Like disabling them from time to time?

If not, then it is possible that some "event" the hardware is looking for (like the SCL line going high/low) is being missed due to line noise.

What you could consider is a "watchdog timer". That is, every couple of seconds a watchdog interrupt fires, and drags the program back into the main loop. Since you have an onboard clock anyway you could work out from that what you need to do next.

Did you try the suggestion of lower pull-up resistors? Over a long cable run the wire resistance might be so great that what should be a nice square wave at the LEDs is turning into a wishy-washy round signal that they are having trouble interpreting.

Those last 2 posts really gave me a lot to look into :fearful:

Perhaps it's time to incorporate an I2C extender into your design. http://ics.nxp.com/products/i2chubs/#Introduction . I'm guessing that if put a scope on the signals they will look like crap!

I would agree. I think the issue is a bad signal.

There was an excellent document on that site:

http://ics.nxp.com/literature/presentations/interface/pdf/debug.i2c.oscilloscope.pdf

People interested in i2c should give it a look.

A wild guess at my freezing problem would be this quote:

The SDA line is stuck low because the slave is stuck in the transmitter mode

So an I2C extender really seems like it would solve my problem. But I couldn't find a product I can just start using : (

Can you recommend a product? An I2C extender breakout board for example?

So the first question is, are you fiddling with interrupts? Like disabling them from time to time?

Nope.

What you could consider is a "watchdog timer".

Researching a bit shows that to be very tricky. Maybe too tricky for me. Unless there is an easy way to do it without a risk of locking up your Arduino or having to create custom bootloaders.

Did you try the suggestion of lower pull-up resistors?

I did. I didn't have 1.5k as suggested, but I did try 2.2k and noticed no difference in stability. I will try 1.5k when I get my hands on them.

I'm considering replacing the ribbon cable with a cat 5 between the lights in case that helps keep noise out. Are there any off the shelf adapters with a cat 5 female on one side and a 0.1" 4 or more input female on the other? Or would making 15 patch points add just as much instability?

Also, would putting decoupling capacitors on clock and gnd and data and gnd between each light do anything? Or would that destroy the signal instead of stabilize it?

Thanks for all your help!

Another interesting quote from the set of slides you found:

Problem: Due to the bus 400 pF
maximum capacitive load limit,
sending commands over wire (80
pF/m) long distances is hard to
achieve

That suggests that cables should be shorter than about 15 feet.

Ribbon cable is especially bad. Due both to capacitance and cross-talk, ribbon cable is not recommended for lengths over about 10 feet.

Any way to reduce the i2c clock speed? A slower signal should give better performance.

[quote author=Nick Gammon link=topic=55243.msg397325#msg397325 date=1300216206]
The endTransmission shouldn't go into a loop for no reason ....[/quote]

Quite the contrary, endTransmission is typically where the wire library freezes when there are bus issues. It sits in a loop waiting for the bus to become ready and this may never happen in some circumstances. A reset of the I2C sub-module (or the microcontroller) is often required to recover from this. Sometimes this involves resetting all I2C nodes as you never know which node holds the bus in error.

The fact that it freezes the Arduino is a library issue and ideally it should never happen irrespective of bus issues. It is also the case however that it will probably never be an issue if there are no hardware/bus related hickups.

One way to approach this is to fix the wire library. The following link offers some details on how this can be accomplished.

http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1283887406

SparksAlot:

What you could consider is a "watchdog timer".

Researching a bit shows that to be very tricky. Maybe too tricky for me. Unless there is an easy way to do it without a risk of locking up your Arduino or having to create custom bootloaders.

It's not tricky. No custom bootloaders needed. This example demonstrates:

#include <avr/wdt.h>

void setup ()
{
  Serial.begin (9600);
  wdt_enable(WDTO_1S);  // reset after one second, if no "pat the dog" received
 }  // end of setup

void loop ()
{
  
Serial.println ("Entered loop ...");

Serial.println ("Point A");
delay (500);
Serial.println ("Point B");
delay (500);

wdt_reset();  // give me another second to do stuff (pat the dog)

Serial.println ("Point C");
delay (500);
Serial.println ("Point D");
delay (500);

while (true) ;   // oops, went into a loop
    
}  // end of loop

This resets the Arduino after 1 second. If you are doing something like flashing LEDs then the occasional pause while it regroups for part of a second won't be too bad.

In the example as shown, you see points A do D printed. Take out the wdt_reset() and it only does A to C (which takes one second).

So you just need the enable in setup, and do a wdt_reset() in the main loop. That way if an I2C communication hangs the program simply restarts automatically.

BenF:
One way to approach this is to fix the wire library.

It went into a loop for a reason. The reason was no interrupt received. But I agree that the suggested fix of adding a timeout would be one way out of it. Still, if you are going to timeout in the middle, and maybe reset the I2C hardware anyway, then the watchdog timer would effectively do that anyway.

It looks like the watchdog timer only has a range of 1-8 seconds?

Is there a way to get more time? I would prefer 3-4 minutes.

I need big lengths of time in my code for motor movements for example. There are many long delays in my code longer than 8 seconds. Plus many little ones that add up.

Would prefer not to bust up my code into little 8 second chunks if possible.

I put in one decoupling capacitor and the 1.5k resistors. It did seem to help, but still get freezing. Just after longer periods :~

I put in the decoupling capacitor before the RTC but after the pull up resistors. Can I put another capacitor before the pull up resistors? That would seem to give a clean signal to the i2c clock and data lines maybe?

Can I put as many capacitors as I like all over? Or are there problems in doing so?

To ask again, would putting decoupling capacitors on clock and gnd and data and gnd between each light do anything? Or would that destroy the signal instead of stabilize it?

And again, does anyone know where to buy an i2c extender breakout board?

Thanks : )

I wouldn't put capacitors on the SDA/SCL lines at all. The whole point of the pull-ups is to get a nice square wave, like this:

The capacitor would round it off. Possibly your long cable runs are having the same effect anyway, so you don't want to add to it. The capacitors would be useful on the Vcc lines at various places (like, each LED) to ensure that the chip gets a clean power signal. But not the data lines.

As for the timer, one approach is to simply reset it in any loop where you know "things are happening" (like moving motors). You don't need to break them into 8 second chunks.

Another would be to simply enable/disable it when you actually are doing the I2C stuff. Something like this, but I haven't tested it:

wdt_enable(WDTO_1S); 
Wire.endTransmission ();
wdt_disable ();

Since it hangs in the endTransmission, that is the only place you really care about being "saved" from.

Hello,
I encountered some I2C problems, too. The reason was a 5m cable and its capacity.
In your case, I would decouple the I2C between the RTC (internal) and the LED part (external). There are very nice possibilities to do it with, for example, the ISL3301. So you can have I2C lines buffered and this may help to keep the internal signals clean.
The reason of I2C problems is oftenly that you have to charge the cable via pull ups, that means passive. But don't make the pullups too small, it's not the optimal solution. Accelerators such as the ISL use current sources for that job. This is much better...

SparksAlot:
I put in one decoupling capacitor and the 1.5k resistors. It did seem to help, but still get freezing. Just after longer periods :~

I put in the decoupling capacitor before the RTC but after the pull up resistors. Can I put another capacitor before the pull up resistors? That would seem to give a clean signal to the i2c clock and data lines maybe?

Can I put as many capacitors as I like all over? Or are there problems in doing so?

To ask again, would putting decoupling capacitors on clock and gnd and data and gnd between each light do anything? Or would that destroy the signal instead of stabilize it?

Just to reiterate -- capacitance is the probable problem, not the solution.

The reason why you want to put capacitance on the power lines is that to smooth them out. Nice steady power is good. Any spikes or changes are harmful noise -- you want to get rid of them.

With your signal, on the other hand, you don't want to smooth out the transitions. The transitions are not noise; they are signal. Capacitance will smooth the transitions and harm your signal. In fact, your 5 meters of ribbon cable probably exceeds the 400 pF maximum capacitance that the i2c standard permits. So although the device works, there are occasional problems.

The problem seems likely to be either noise or sloppy signal, or both. From the fact that your device freezes every now and then, I suspect noise. Ribbon cables are not good for noise. Using a Cat5 cable with signal and ground wires in twisted pairs will help combat noise.

But the rather long, rather high-capacitance ribbon cable seems to point to sloppy signal instead of noise. If you have access to an oscilloscope, that might tell the tale. It would be interesting to see if you have a squarish type signal or more of a sawtooth.

As people have mentioned, a buffer might help. Sorry that I don't know of any i2c buffer breakout boards. Good luck with the project.

You had asked about RJ45 connector for the receive end?

6-pin & 8-pin: