Why am I running out of memory? i2c

Hello, I am working on a project where an Arduino Uno gathers bytes from an SD card and sends the bytes to a Rainbowduino which is an LED driver that uses the Arduino bootloader, and I’m sending the info over i2c. The Rainbowduino then lights the LEDs for each frame.The program is working fine for the most part (although it’s slower than I’d like), except after 34 frames the LEDs flash either a random color or they just go off until all the frames are done, and then it restarts the animation. I assume this is because I am running out of RAM but I’m not sure why. The bytes obtained from the SD card should be overwriting the same variables, so why is the program acting up? I have included the basic elements of my code. Note that I increased the i2c buffer to 98 from 32. Also, if you have any suggestions for increasing speed I’d greatly appreciate it.

Host (Arduino Uno)

#include <Wire.h>
#include <SD.h>

int frames; //this number is obtained from the SD card. It's the number of frames.
byte rgb[192]; //pretend these 192 bytes are obtained from the SD card for every frame.

void setup(){
  Wire.begin();
  pinMode(10,OUTPUT); //ChipSelect pin for the SD
}

void loop(){
for(int k=0;k<frames;k++){
for(byte j=0;j<192;j++){
 rgb[j]=rgbByte(j,k);
 Serial.println(rgb[j]);
      }
    Wire.beginTransmission(2);
Wire.write(1);
for(byte j=0;j<96;j++)
Wire.write(rgb[j]);
Wire.endTransmission();   
//delay(100);
Wire.beginTransmission(2);
Wire.write(3);
for(byte j=96;j<192;j++)
Wire.write(rgb[j]);

Wire.endTransmission();
}    
}

Slave (Rainbowduino):

#include "Wire.h"
#include <Rainbowduino.h>
byte rgb[192];
void setup(){
  Rb.init();
  Wire.begin(2);
  delay(2000);
  Wire.onReceive(expansionReceive);
}
  
void loop(){
}

void expansionReceive(int howMuch){
int q=Wire.read();
if(q==1){
  for(int j=0;j<96;j++)
  rgb[j]=Wire.read();
}
  
else if(q==3){
  for(int j=96;j<192;j++)
  rgb[j]=Wire.read();


Rb.blankDisplay();
byte y=7; //this is what sets the display, ignore this probably.
byte x=7;
  for(int j=0;j<192;j+=3){
Rb.setPixelXY(y,x,rgb[j],rgb[j+1],rgb[j+2]);

  if(y==0){
  y=8;
  x--;
  }
  y--;
  }
}
}

I changed the buffer size to 32 and adjusted the program accordingly and I am still getting the same issue. Since that should have dramatically reduced RAM consumption I'm not really sure what the problem is.

How are you reading the bytes from the sd card. Also you should NEVER use SD.h it is an out of date version of sdfatlib that the arduino people hacked to make it slower and ruin your sd card by flushing the buffer after every byte write. Please use sdfatlib instead https://code.google.com/p/sdfatlib/

Thanks I didn't know that. I'll check that library out. Here's the function for getting the bytes from the SD:

byte rgbByte(byte j,int k){//j is for each frame, k is overall
  File myFile;
  myFile= SD.open("matrix.txt");
  myFile.seek(0+5*j+960*k);
  byte x=myFile.read();
  x -= 48;
  x *= 100;
  myFile.seek(1+5*j+960*k);
  byte y=myFile.read();
  y -= 48;
  y *= 10;
  myFile.seek(2+5*j+960*k);
  byte z=myFile.read();
  z -= 48;
  myFile.close();
  return (x+y+z);
}

matrix.txt looks like this:

000
012
155
025
240
192
so 3 bytes of info and 2 bytes of formatting per color per pixel

It would be a lot easier if you would just post or attach your whole code, instead of just snip its.

Sorry. This is the code I just modified for the 32 buffer. The function for obtaining the number of frames is really messy. On the SD card there is a file called frames.txt that has the number of frames only, and matrix.txt that has all the bytes.

Host

#include <Wire.h>
#include <SD.h>

  int frames,frame1,frame2,frame3,frame4;
  byte rgb[192];
void setup(){
  Serial.begin(9600);
  Wire.begin(1);
  pinMode(10,OUTPUT);
  if(!SD.begin(10))
  Serial.println("fail");
  frames=framers();
  Serial.println(frames);
}


void loop(){
for(int k=0;k<frames;k++){
for(byte j=0;j<192;j++){
 rgb[j]=rgbByte(j,k);
 Serial.println(rgb[j]);
      }
      for(int w=0;w<6;w++){
    Wire.beginTransmission(2);
Wire.write(w);
for(byte j=32*w;j<32*(w+1);j++)
Wire.write(rgb[j]);
Wire.endTransmission();   
      }
}    
}

byte rgbByte(byte j,int k){//j is for each frame, k is overall
  File myFile;
  myFile= SD.open("matrix.txt");
  myFile.seek(0+5*j+960*k);
  byte x=myFile.read();
  x -= 48;
  x *= 100;
  myFile.seek(1+5*j+960*k);
  byte y=myFile.read();
  y -= 48;
  y *= 10;
  myFile.seek(2+5*j+960*k);
  byte z=myFile.read();
  z -= 48;
  myFile.close();
  return (x+y+z);
}

int framers(){
  int f0,f1,f2,f3,f4,f5,x;
  File myFile;
  myFile = SD.open("FRAMES.TXT");
  if(myFile)
  Serial.println("Frames loaded");
  x=myFile.available();
  switch (x){
  case 1:
  f1=myFile.read()-48;
  f0=f1;
  break;
  case 2:
  f1=myFile.read()-48;
  myFile.seek(1);
  f2=myFile.read()-48;
  f0=f1*10+f2;
  break;
  case 3:
  f1=myFile.read()-48;
  myFile.seek(1);
  f2=myFile.read()-48;
  myFile.seek(2);
  f3=myFile.read()-48;
  f0=f1*100+f2*10+f3;
  break;
  case 4:
  f1=myFile.read()-48;
  myFile.seek(1);
  f2=myFile.read()-48;
  myFile.seek(2);
  f3=myFile.read()-48;
  myFile.seek(3);
  f4=myFile.read()-48;
  f0=f1*1000+f2*100+f3*10+f4;
  break;
   case 5:
  f1=myFile.read()-48;
  myFile.seek(1);
  f2=myFile.read()-48;
  myFile.seek(2);
  f3=myFile.read()-48;
  myFile.seek(3);
  f4=myFile.read()-48;
  myFile.seek(4);
  f5=myFile.read()-48;
  f0=f1*10000+f2*1000+f3*100+f4*10+f5;
  break;
  }
  myFile.seek(0);
  return f0;
  myFile.close();
}

slave

#include "Wire.h"
#include <Rainbowduino.h>
//byte r[64],g[64],b[64];
byte rgb[192];
int w=0;
void setup(){
  Rb.init();
  Wire.begin(2);
  delay(2000);
  Wire.onReceive(expansionReceive);
}
  
void loop(){
  //Rb.blankDisplay();
  //Wire.requestFrom(1,1);

}

void expansionReceive(int howMuch){
 // Rb.blankDisplay();
  //for(int q=0;q<2;q++)
int q=Wire.read();
if(q==0){
  for(int j=0;j<32;j++)
  rgb[j]=Wire.read();
}
  
else if(q==1){
  for(int j=32;j<64;j++)
  rgb[j]=Wire.read();
}
else if(q==2){
  for(int j=64;j<96;j++)
  rgb[j]=Wire.read();
}else if(q==3){
  for(int j=96;j<128;j++)
  rgb[j]=Wire.read();
}else if(q==4){
  for(int j=128;j<160;j++)
  rgb[j]=Wire.read();
}else if(q==5){
  for(int j=160;j<192;j++)
  rgb[j]=Wire.read();

Rb.blankDisplay();
byte y=7;
byte x=7;
  for(int j=0;j<192;j+=3){
Rb.setPixelXY(y,x,rgb[j],rgb[j+1],rgb[j+2]);

  if(y==0){
  y=8;
  x--;
  }
  y--;
  }
}
//for(int j=0;j<64;j++)
//rgb[j]=Wire.read();//+96*q]=Wire.read();

 // for(int j=0;j<64;j++){
 // Rb.setPixelXY(0,y,rgb[j],0,0);//rgb[j+64],rgb[j+128]);
 // y++;
//if(y==64)
//y=0;
//}


}

I don't know where to begin no words can describe how inefficient and slow and bad is your code. I don't mean to be rude or anything but first of all you are opening seeking and closing the file every-time you want a byte don't do that. Just open the file at the beginning of program and keep reading from file no need to open seek close every time. Second of all you don't want to store the numbers in plain text that makes it slower because it requires conversion code so that makes it slower just store the numbers are raw binary data you can use a hex editor for that remember the avr-gcc outputs little endian code Also try to make the data as pre-processed as possible for example if the numbers needs to be divided by 2 instead of doing that in the arduino have your data already divided by 2 and so on. SD cards have so much space anyways. Third of all you are still using SD.h don't use that use the latest version of sdfatlib instead https://code.google.com/p/sdfatlib/downloads/list
No wonder your code runs slower than you like.

Thanks. I don't really know what you mean by storing it as raw binary data...I made a GUI in Processing for designing frames that outputs all the bytes, and I assume you're talking about saving it as FFFFFF instead of 255 255 255, but what kind of file would I export to? Also, I integrated SdFatLib into my program but I can't find what the equivalent function to seek is. I am trying to do the same thing I had before where I read, seek, read, seek until all the bytes per frame are read, unless there is a better way.


Here are two different ways to store the number FFFFFFFF
The one on the left is stored using ascii notice you can read it in the box to the far right it takes twice as much memory to store and requires conversion code to be ran on the arduino
The one on the right is stored as raw data so to say it is already in a format readable by the micro-controller. You don't have to do any conversion code it is already in a format usesable by the arduino.
Also note that I would recommend against a 32-bit variable (above is just an example) on the arduino unless you need that large of size. Try to use the smallest variable that can hold your data. If you data is 0-255 just use an unsigned 8bit variable.
Also to avoid seeking every-time just make sure you have everything stored in the correct order to seek you use the seekSet() function in sdfatlib
So here is how you would avoid seeking every-byte
Open file
Read bytes using read function
Send to device
Repeat until end of file
(optional) seek to beginning of file and repeat all steps except opening the file. If you want to use a different file close the current file and open another.
I think I just realized your problem you seem to think that when you call the read function it will read the same bytes every-time unless you call the seek function that is not true what happens when you call read is it automatically increments the file position so the next time you call read it will read the bytes right after the bytes you read last time.

But I would store FFFFFF, for example, in a txt document, right? Aside from that, I got it working XD. The frames no longer mess up after 34, so the primary issue was the standard SD library. Thanks for all your help, I really appreciate it, and i apologize for my programming naivety but I'm working on it :wink:

If you can see it in a text editor you are doing it wrong use a hex editor. However you got it working and if you are happy don't worry about it for now.