LED Brightness from pixel data [Firmata] very slow

Forum 2005-2010 (read only) Software Interfacing
1

Hey there,

I'm working on a project to take pixel data from a web cam stream, and use it to adjust the brightness of an 8x8 matrix of LEDs. I'm currently using a 2D array in processing to store the pixel data from a 64 x 64 webcam stream, and then send it to the LEDs via Firmata. I was working on a basic version with a row of 8 LEDs.

The problem I'm running in to is that the brightness change is fairly small, and is extremely choppy, and not dynamic.

The LEDs are 8 super brights, running through 330 ohm resistors, to 8 PWM pins on an Arduino Mega.

My questions:

Is Firmata a particularly slow process? Is it better to figure out how to bring the data into an Arduino program through serial?

If so, how do I specify bytes? I've been looking at the LED dimmer tutorial. The tutorial only has 1 LED, how do I specify that the pixel from grid[0][1] should go to ledPin1 and so forth?

Is there a better way I could be going about this?

Thank you!

2

Also: can you recommend a way to better utilize my PWM pins? Clearly I'm running out of pins, is there a better way to do this?

3

Is Firmata a particularly slow process?

Not particularly, but it is certainly not optimized for your use.

The data is sent as strings, which means that the data needs to be converted to strings, send as strings, and converted back to numbers on the Arduino.

You need to send 64 bytes of data for each frame. Converting all these numbers to strings, and back again takes a fair amount of time.

Dump Firmata, and send each packet as 66 bytes - a start of packet marker, 64 bytes, and an end-of-packet marker.

On the Arduino, ignore any packets that do not contain 64 values.

The problem I'm running in to is that the brightness change is fairly small

That isn't a Firmata issue. Either the data you are sending is bad, or the way you are using the data is.

and is extremely choppy, and not dynamic.

This may or may not be a Firmata issue. Is Processing analyzing the data fast enough?

If so, how do I specify bytes?

Where? In Processing or on the Arduino?

how do I specify that the pixel from grid[0][1] should go to ledPin1 and so forth?

Using arrays on the Arduino, too.

4

I solved my problem, sort of. I was writing to Arduino inside a couple of nested for loops, like so:

for (int i = 0; i < cols; i++) {
// Begin loop for rows
for (int j = 0; j < rows; j++) {
grid*[j].display();*

  • grid0num = grid[0][0].value();*
  • // myPort.write(grid0num);*
  • arduino.analogWrite(ledPin0, grid0num);*
  • grid1num = grid[0][1].value();*
  • // println(grid1num);*
  • // myPort.write(grid1num);*
  • arduino.analogWrite(ledPin1, grid1num);*
    Once I removed the write to outside the for loop, the performance increased dramatically and I now have LEDs that get brighter/dimmer depending on what the webcam is seeing.
    This works fine for 8 LEDs, and only for about 5 minutes, until the Arduino seems to get overwhelmed by data and the LEDs stop responding.
    The main problem I'm running in to now is how to create this effect for 64 LEDs. I used 8 PWM pins, and the mega only has 13, so I can't hardwire it.
    It's been suggested to me to use shift registers or a CD4067 analog multiplexer, which I suppose means I have to get serial to work.
    I'm using the simple dimmer tutorial from the Arduino playground. When I open serial I get a character, but the brightness of the LED does not change.
    Here is my code (I know its awful):
    ```
    import cc.arduino.;
    import processing.serial.*;

// Learning Processing
// Daniel Shiffman
// http://www.learningprocessing.com

// Example 13-10: Two-dimensional array of objects

// 2D Array of objects
Cell[][] grid;
int c = 0;
int i = 0;
// Number of columns and rows in the grid
int cols = 8;
int rows = 8;
Arduino arduino;
import processing.video.*;
int videoScale = 8;
Serial myPort;

int inByte;
int ledPin0 = 2;
int ledPin1 = 3;
int ledPin2 = 4;
int ledPin3 = 5;
int ledPin4 = 6;
int ledPin5 = 7;
int ledPin6 = 8;
int ledPin7 = 9;

int ledPin8 = 14;
int ledPin9 = 15;
int ledPin10 = 16;

int grid0num = 0;
int grid1num = 0;
int grid2num = 0;
int grid3num = 0;
int grid4num = 0;
int grid5num = 0;
int grid6num = 0;
int grid7num = 0;

int grid8num = 0;
int grid9num = 0;
int grid10num = 0;

//int cols, rows;

Capture video;

void setup() {
 size( 64,64 );
 grid = new Cell[cols][rows];

video = new Capture(this,cols,rows,30);

// The counter variables i and j are also the column and row numbers
 // In this example, they are used as arguments to the constructor for each object in the grid.
 for (int i = 0; i < cols; i ++ ) {
   for (int j = 0; j < rows; j ++ ) {
     // Initialize each object
     grid[i][j] = new Cell(ivideoScale,jvideoScale,videoScale,videoScale,i + j);
   }
 }
 //println("available serial ports:");
 println(Arduino.list());
  myPort = new Serial(this, Serial.list()[1], 9600);
/*  arduino = new Arduino(this, Arduino.list()[1], 57600);
 arduino.pinMode(ledPin0, arduino.OUTPUT);
 arduino.pinMode(ledPin1, arduino.OUTPUT);
 arduino.pinMode(ledPin2, arduino.OUTPUT);
 arduino.pinMode(ledPin3, arduino.OUTPUT);
 arduino.pinMode(ledPin4, arduino.OUTPUT);
 arduino.pinMode(ledPin5, arduino.OUTPUT);
 arduino.pinMode(ledPin6, arduino.OUTPUT);
 arduino.pinMode(ledPin7, arduino.OUTPUT);

arduino.pinMode(ledPin8, arduino.OUTPUT);
 arduino.pinMode(ledPin9, arduino.OUTPUT);
 arduino.pinMode(ledPin10, arduino.OUTPUT);
 */
}

void draw() {

//while (myPort.available () > 0) {
 //int inByte = myPort.read();
 //println(inByte);
 //r }

// Read image from the camera
 if (video.available()) {
   video.read();
 }
 video.loadPixels();

// Begin loop for columns
 for (int i = 0; i < cols; i++) {
   // Begin loop for rows
   for (int j = 0; j < rows; j++) {
     grid[i][j].display();

grid0num = grid[0][0].value();
     // myPort.write(grid0num);
     // arduino.analogWrite(ledPin0, grid0num);

grid1num = grid[0][1].value();
     //  println(grid1num);
     //  myPort.write(grid1num);
     // arduino.analogWrite(ledPin1, grid1num);

grid2num = grid[0][2].value();
     //  println(grid2num);
     // myPort.write(grid2num);
     // arduino.analogWrite(ledPin2, grid2num);

grid3num = grid[0][3].value();
     //  println(grid3num);
     //myPort.write(grid3num);
     //  arduino.analogWrite(ledPin3, grid3num);

grid4num = grid[0][4].value();
     //myPort.write(grid4num);
     //  arduino.analogWrite(ledPin4, grid4num);

grid5num = grid[0][5].value();
     //myPort.write(grid5num);
     //   arduino.analogWrite(ledPin5, grid5num);

grid6num = grid[0][6].value();
     //myPort.write(grid6num);
     //  arduino.analogWrite(ledPin6, grid6num);

grid7num = grid[0][7].value();
     //myPort.write(grid7num);
     //  arduino.analogWrite(ledPin7, grid7num);
     
       grid8num = grid[1][0].value();
      //myPort.write(grid8num);
     // arduino.analogWrite(ledPin2, grid0num);
     
     
      grid9num = grid[1][1].value();
     // myPort.write(grid9num);
      grid10num = grid[1][2].value();
     // myPort.write(grid10num);
     
      /*
      int grid11num = grid[1][3].value();
      myPort.write(grid11num);
      int grid12num = grid[1][4].value();
      myPort.write(grid12num);
      int grid13num = grid[1][5].value();
      myPort.write(grid13num);
      int grid14num = grid[1][6].value();
      myPort.write(grid14num);
      int grid15num = grid[1][7].value();
      myPort.write(grid15num);
      int grid16num = grid[2][0].value();
      myPort.write(grid16num);
      int grid17num = grid[2][1].value();
      myPort.write(grid17num);
      int grid18num = grid[2][2].value();
      myPort.write(grid18num);
      int grid19num = grid[2][3].value();
      myPort.write(grid19num);
      int grid20num = grid[2][4].value();
      myPort.write(grid20num);
      int grid21num = grid[2][5].value();
      myPort.write(grid21num);
      int grid22num = grid[2][6].value();
      myPort.write(grid22num);
      int grid23num = grid[2][7].value();
      myPort.write(grid23num);
      int grid24num = grid[3][0].value();
      myPort.write(grid24num);
      int grid25num = grid[3][1].value();
      myPort.write(grid25num);
      int grid26num = grid[3][2].value();
      myPort.write(grid26num);
      int grid27num = grid[3][3].value();
      myPort.write(grid27num);
      int grid28num = grid[3][4].value();
      myPort.write(grid28num);
      int grid29num = grid[3][5].value();
      myPort.write(grid29num);
      int grid30num = grid[3][6].value();
      myPort.write(grid30num);
      int grid31num = grid[3][7].value();
      myPort.write(grid31num);
      int grid32num = grid[4][0].value();
      myPort.write(grid32num);
      int grid33num = grid[4][1].value();
      myPort.write(grid33num);
      int grid34num = grid[4][2].value();
      myPort.write(grid34num);
      int grid35num = grid[4][3].value();
      myPort.write(grid35num);
      int grid36num = grid[4][4].value();
      myPort.write(grid36num);
      int grid37num = grid[4][5].value();
      myPort.write(grid37num);
      int grid38num = grid[4][6].value();
      myPort.write(grid38num);
      int grid39num = grid[4][7].value();
      myPort.write(grid39num);
      int grid40num = grid[5][0].value();
      myPort.write(grid40num);
      int grid41num = grid[5][1].value();
      myPort.write(grid41num);
      int grid42num = grid[5][2].value();
      myPort.write(grid42num);
      int grid43num = grid[5][3].value();
      myPort.write(grid43num);
      int grid44num = grid[5][4].value();
      myPort.write(grid44num);
      int grid45num = grid[5][5].value();
      myPort.write(grid45num);
      int grid46num = grid[5][6].value();
      myPort.write(grid46num);
      int grid47num = grid[5][7].value();
      myPort.write(grid47num);
      int grid48num = grid[6][0].value();
      myPort.write(grid48num);
      int grid49num = grid[6][1].value();
      myPort.write(grid49num);
      int grid50num = grid[6][2].value();
      myPort.write(grid50num);
      int grid51num = grid[6][3].value();
      myPort.write(grid51num);
      int grid52num = grid[6][4].value();
      myPort.write(grid52num);
      int grid53num = grid[6][5].value();
      myPort.write(grid53num);
      int grid54num = grid[6][6].value();
      myPort.write(grid54num);
      int grid55num = grid[6][7].value();
      myPort.write(grid55num);
      int grid56num = grid[7][0].value();
      myPort.write(grid56num);
      int grid57num = grid[7][1].value();
      myPort.write(grid57num);
      int grid58num = grid[7][2].value();
      myPort.write(grid58num);
      int grid59num = grid[7][3].value();
      myPort.write(grid59num);
      int grid60num = grid[7][4].value();
      myPort.write(grid60num);
      int grid61num = grid[7][5].value();
      myPort.write(grid61num);
      int grid62num = grid[7][6].value();
      myPort.write(grid62num);
      int grid63num = grid[7][7].value();
      myPort.write(grid63num);
      /
   }
 }
 /
 arduino.analogWrite(ledPin0, grid0num);
 arduino.analogWrite(ledPin1, grid1num);
 arduino.analogWrite(ledPin2, grid2num);
 arduino.analogWrite(ledPin3, grid3num);
 arduino.analogWrite(ledPin4, grid4num);
 arduino.analogWrite(ledPin5, grid5num);
 arduino.analogWrite(ledPin6, grid6num);
 arduino.analogWrite(ledPin7, grid7num);
 */
 println(grid0num);
 myPort.write(grid0num);

}
//}

// A Cell object

class Cell {

// A cell object knows about its location in the grid as well as its size with the variables x, y, w, h.
 float x,y;   // x,y location
 float w,h;   // width and height

// Cell Constructor
 Cell(float tempX, float tempY, float tempW, float tempH, float tempAngle) {
   x = tempX;
   y = tempY;
   w = tempW;
   h = tempH;
 }

void display() {

int pixelX = int(x)/videoScale;
  int pixelY = int(y)/videoScale;
  int loc = pixelX+pixelY*video.width;
  color c = video.pixels[loc];
  float value = brightness(c);
  //println(value);
  fill(brightness(c));
  rect(x,y,w,h);
}

int value(){
  int pixelX = int(x)/videoScale;
  int pixelY = int(y)/videoScale;
  int loc = pixelX+pixelYvideo.width;
  color c = video.pixels[loc];
  int value = int(brightness(c));
 return value;
 
}
```

5

You could put the pin numbers in an array, too.

Why are there 11 pin numbers? Why are there 11 grid numbers?

       int grid50num = grid[6][2].value();
       myPort.write(grid50num);

Simpler to do this:

myPort.write(grid[6][2].value());
6

It's been suggested to me to use shift registers or a CD4067 analog multiplexer,

No you don't want to do this. Controlling brightness with a shift register is not possible with all the other work the arduino has to do. Using an analogue multiplexer to drive an LED is something only an idiot would do.

You want something that will take the load off the processor, something like a few TLC5940s would do.