ShiftMatrix, Dreaming Big Video Display

Yeah, for my own sanity I kept it the same. Essentially my Processing program sends a giant byte array down the serial line in RGB. The Arduino catches it and puts it (in the same order) directly into the array that is setup for the LED brightness values.

Wow man, this is looking quite amazing already! I'm busy the the same thing apart from the controlling, im using 8 74hc595 shift registers and arranged them like this:

[rows 1-8 v+][rows 9-16V+]
[cols red 1-8 gnd][cols red 9-16 gnd]
[cols green 1-8 gnd][cols green 9-16 gnd]
[cols blue 1-8 gnd][cols blue 9-16 gnd]

However im still not getting the code working the way i want it to but im understanding it more and more and this thread is also really helpfull as it gives you a lot of info!

Thanks Xforce. Its good to know that this information is helping others. I don't think that what I am doing is particularly hard its just very time consuming. I have a basic grasp of electronics and I try to be very thorough. (I have been planning to do this for over a year) All the heavy lifting of making this work has been done by ElcoJacobs and the ShiftPWM\ShiftMatrixPWM library's that he created. I think if it weren't for him I would still be trying to write a basic 8x8 pwm matrix routine. His library means you can just add the components and tell it what to do. This has allowed me to focus on getting the information I want streamed to the device. As long as I send it the right info, it does exactly what I want. In my case this meant grabing video frames and converting them to a serial data stream and sending it to the Arduino. And really its been as simple as that.

All I am trying to do is show others how I achieved it, hopefully with enough info to replicate it if they would like to. What I have accomplished with this project has only been possible due to the huge amount of information on these forums and in the tutorials that people have made. At some stage I will draw up a "Step by step" of how this all went together. I have my Proccessing code commented and nearly ready to share (I think :)) which should be the final piece to having the ability to make what I have made so far.

With regard to my code for processing, I have two different sketches. One for sending a RGB colour byte stream and one for sending an averaged mono byte stream. Which would you prefer to see? Or would you like both? Or should I put a switch in the code so that it could do either?

Hmmmmm it always seems like there is something else to do to the code! Haha
If there is anything else you guys are after just let me know. I will try and help if I can.

Hey man,
I'd be great if you could do the switch one so i can get an idea of your way or programming and thinking. and i think im also gonna focus more on trying to get Elco Jacob's library to work cus that would probably be easier than trying to write some routine myself ^^. I don't mind on how you'll send me the code, via pm or you post it here, you decide (which is rather logical as you're the one who made the processing sketch:P).

xforce30164:
I'd be great if you could do the switch one so i can get an idea of your way or programming and thinking.

Yeah thats what I decided on so I spent some time reworking it so that it would work either way. I also added a switch so you could just run it onscreen and not actually output the serial, this made testing parts of the code much quicker. I have tried to comment as fully as possible, I have probably gone overboard :D. Let me know what you think. The Arduino part of the code is already up in other posts in this thread.

/*
* ShiftMatrixPWM Serial Video Transmit for Processing
* by Richard Black
* Built-in video library replaced with gsvideo by Andres Colubri
* Get it from: http://gsvideo.sourceforge.net/
*
* Loads a video file and downscales the frames and sends them
* via Serial to an Arduino setup with the ShiftMatrixPWM Library.
* For the Arduino, found at: http://arduino.cc/forum/index.php/topic,66988.0.html
* This displays three images in the window. The original, the down
* scaled and the final output. This code has been setup to work with
* video's in 4:3 ratio but you could adapt it for other ratios.
* It is currently set to output 32 levels of brightness. This can be changed.
*/

import codeanticode.gsvideo.*;
import processing.serial.*;

//SETTINGS START HERE!!!!!!!!!!
//currently I am building a square matrix. I left seperate height and width
//for future possible use.
boolean ColourMatrix = false; // true for colour, false for Mono
boolean OutputEnabled = false; // true for serial out, false no serial out(on screen only)
int mpHigh = 64; //Number of Rows High the Matrix is.
int mpWide = 64; //Number of Columns wide the Matrix is.
int movRate = 40; //Set the number of frames per sec to process
String comPort = "COM5"; //Set what port the Arduino is on
String movFile = "video.mp4"; //set which file to play. This needs to be in the projects data folder
int brightLevel = 32; //This is the number of levels required. On the Arduino this would be 0-31
//using this code bightLevel should be a multiple of 8 so it can be cleanly divided into 256 as further down
//SETTINGS END HERE!!!!!!!!!!!!

Serial matrixPort;
GSMovie theMovie; //Creates the GSvideo Movie object.
PImage smallFrame = new PImage(mpWide, mpHigh);
color pixelclr; //this is for color setup
int pixelbright; //this is for mono color setup
byte [] myFrame;
PFont font;
int divider = 256 / brightLevel; //This allows calculating the brightness levels in the code

void setup() {
 size(1000, 400, P3D);
 if (ColourMatrix==true){
  myFrame = new byte[mpWide*3*mpHigh];
} else {
  myFrame = new byte[mpWide*mpHigh];
}
 matrixPort = new Serial(this, comPort, 250000);// Initialise com port replace with your speed
 frameRate(movRate);//framerate set above in settings
 noStroke();
 background(0);
 theMovie = new GSMovie(this, movFile); //Initialises instance of the movie
 theMovie.loop(); // I have it looping. You can comment this out if you don't want looping
 //theMovie.play(); // Use this to have the movie only play once. No Repeat.
 ellipseMode(CORNER);
 font = loadFont("DejaVuSans-24.vlw"); //Change to whatever font file you have otherwise, comment out
 //comment out the next five lines if there is no font or above is commented out
 textFont(font);
 fill(0, 102, 153);
 text("Original", 15, theMovie.height+20); // Label
 text("Down Scaled", theMovie.width+20, (smallFrame.height*2)+20);// Label
 text("Output", theMovie.width+20, (height/2)-10);// Label
}

// Read new values from movie
void movieEvent(GSMovie m) {
  m.read();// Reads in the new Frame of the movie
  //This next line takes the movie frame and resizes it into a smaller PImage object.
  smallFrame.copy(m, m.width/8, 0, int(m.width-(m.width/4)), m.height, 0, 0, smallFrame.width, smallFrame.height);
  //This also only gets the center part of the video as the ratio is not a perfect
  //square. The left and right edges get trimed. Currently this is set for 4:3 ratio to square (ie the matrix)
}

// Display values from movie
void draw()  {
  image(smallFrame,theMovie.width+10,0,smallFrame.width*2,smallFrame.height*2); //Draws the small Frame
  //image(theMovie,0,0,theMovie.width/2,theMovie.height/2); //Draws the full movie Frame, half size optional
  image(theMovie,0,0);//Draws the full movie Frame
  for (int y = 0; y < smallFrame.height;y++) {
    for (int x = 0; x < smallFrame.width;x++) {
      if (ColourMatrix==false){
        pixelbright = int(brightness(smallFrame.pixels[(y*smallFrame.width)+x])/divider);
        myFrame[(y*smallFrame.width)+x]=byte(pixelbright);
        fill(pixelbright*divider);
      } else {
        pixelclr = smallFrame.pixels[(y*smallFrame.width)+x];
        if (red(pixelclr)/divider < 1) {
          myFrame[(y*smallFrame.width*3)+(x*3)]=byte(0);
        } else {
          myFrame[(y*smallFrame.width*3)+(x*3)]=byte(int(pow(31,(red(pixelclr)/divider)/31)));
        }
        if (green(pixelclr)/divider < 1) {
          myFrame[(y*smallFrame.width*3)+(x*3)+1]=byte(0);
        } else {
          myFrame[(y*smallFrame.width*3)+(x*3)+1]=byte(int(pow(31,(green(pixelclr)/divider)/31)));
        }
        if (blue(pixelclr)/divider < 1) {
          myFrame[(y*smallFrame.width*3)+(x*3)+2]=byte(0);
        } else {
          myFrame[(y*smallFrame.width*3)+(x*3)+2]=byte(int(pow(31,(blue(pixelclr)/divider)/31)));
        }
        fill((red(pixelclr)/divider)*divider,(green(pixelclr)/divider)*divider,(blue(pixelclr)/divider)*divider);
      }
      //The ellipse is for drawing the supposed output using the colour/brightness from the Fill above
      ellipse(x*((width/4)/mpWide)+(theMovie.width+10),(height/2)+(y*((height/2)/mpHigh)),((width/4)/mpWide)-1,((height/2)/mpHigh));
    }
  }
  //println(myFrame); //used to print frame data to the output panel, good for testing
  if (OutputEnabled==true){
    matrixPort.write('~'); // start Character. If using 127 levels you will
    //need to change the start Character to some higher ascii value.
    matrixPort.write(myFrame); // the frame data
  }
}

So next part for me is making it split the output into sections so I can run a modular matrix.
But this does run my 16x16 RGB matrix just as is. Hope this makes sense to people. :slight_smile:

Thanks for the code and all the comments, when i've got the time (its busy again at the university ^^) ill look through it and let you know what think but i'm quite sure ill understand how it will work, seeying the amount of comments! =).

i try to transfer my frames data same as you from processing (but the pwm process not based on the shiftmatrix library) and i have trouble to get the right data with arduino because i think my pwm interrupt process is bothering the Serial interrupt process. (have missing bytes etc.)
dont you experienced this or any kind of trouble related to the above listed issue?

Hello adamtoth,

It seems I forgot to mention that I am sending the data at a higher rate then standard. This was worked out via trial and error to find a stable rate that worked for me.

I found recieving data to be a little tricky. I would suggest trying the method I have used (earlier in the thread) that had a counter incrementing with each byte that came in.

If this doesn't help, I would recommend starting a thread and posting up some of the code you are using.

milamber:
I am now ready to move out of testing and to start building the large scale matrix itself. As I said before I am planning to go 64x64 mono. But I am scared that it will be huge(I was planning to use Peg Board as a pre-drilled base board) as it is looking like it will be about 2m x 2m in size. I might chicken out a bit and bring it back to 48x48 mono.....just not sure. Pricing lots of LED's now. I don't suppose anyone knows a good source for 4500 LED's at a cheap price, for delivery to Australia. Other than Ebay that is.
So any other comments or queries?

Great thread. I'm working on quite an ambitious project myself and I will definitely be coming back to this thread for advice. Thanks for sharing and for the videos!

If you're ordering 4500 LEDs, maybe you should look into just ordering them directly from the factory in China through alibaba.com
I think the minimum order is 10,000 with most places...but something to look into.... maybe you could just sell the rest on eBay.

Well as usual its a long time since I posted last (real life gets in the way).

Recently at my local electronics shop (Jaycar) I was pleased to find they had a range of "Arduino like" devices and add on shields/hardware. They all seemed to be from the same company, FreeTronics. I haven't heard of them before but one of the products caught my eye, it was call DMD (Dot Matrix Display). 32x16 Red Dot Matrix Display | Freetronics

This is a self contained unit with 512 LEDs (32x16) Mono colour (Red in this case but apparrently more colours are coming). So I bought it and I am trying to find time to have a look under the hood of the demo code. It seems the product is only new but it seems well constructed and the LEDs are very bright. The demo library is very easy to use but unfortunately it does not support PWM (Damn!) but from the notes in the library it seems that its is possible to get PWM implemented. It just hasn't been done yet. So with that in mind I am going to have a play with the library. If I can get PWM to work then this is a good unit that is readily available and I am happy with the price. And it will DRASTICALLY reduce build time. Which for me at the moment is a huge plus as I don't have much time.

I am still aiming for 64 x 64 which means I would need 8 of these boards...that is on the order of $300 but I am able to spread out that purchase and buy them 1 or 2 at a time. Where as if I were to try and "Make" these myself it would be a little cheaper but would require A LOT more time and I would have to buy all the components up front. So from that point of view that is a trade off I am willing to make.

So has anyone looked at this device or tried to do PWM on boards like this? I am going to go trawl through the forum now and see if there is anything similar that maybe I can adapt. Otherwise my current plan is to try and modify Elco's ShitMatrixPWM library to talk to this device. Not sure how much effort is required but as this device use SPI for the main information and 4 line switches to select the rows I think it should be possible to get it to work. I will let you know how I fair.

Again, Months have passed. But for me they have been productive months.

I am very happy with the output of the Freetronics DMD device mentioned in the above post. I have been able to modify (Butcher :P) the ShiftMatrixPWM library to work on these boards. I have even managed to get it working with 2 boards, 1024 red LED's, fully addressable and running at 50Hz with 32 brightness levels.

The way this hardware works though is very different to what ShiftMatrixPWM expects. I have had to make some major changes under the hood to format the information correctly to work with these displays. So much so I don't think you could still call this ShiftMatrixPWM anymore as it wouldn't work with a normal row/column "hand made" matrix. So I will call what I have made DmdPWM (all original credit goes to Elcojacobs for making a great library to base this off).

With this DmdPWM library now created and tested, I can finally start to order the components to make the display. So the display will consist of:

  • 1 x USB to serial adapter (bought from ebay cheap)
  • 4 x Arduino Duemilanove boards
  • 8 x Dmd's
  • 1 x 5v 40amp Power Supply

Currently I am testing with 1 Duemilanove and 2 Dmd's and using an ATX power supply (Looking to get a bench supply :)).
Here is a video being down scaled and serially sent to the Arduino with 2 Dmd's attached. This is just a fire effect on a loop.

I am still trying to round out the library code and clean it up but if anyone is interested in it, let me know. I haven't had anyone else test it yet. If you have one or a similar display (32x16, 1/4 scan and a zigzag register setup (not sure what to call it)) contact me.

I am curious as to how your project went in the end. Currently I'm trying to have a video feed on a 16x16 matrix...I think there's a long way ahead as I'm trying to get the shiftMatrixPWM working on my 4x4 :roll_eyes: .

Hi nataliew,

Time has not been my friend lately. Sadly I have not gotten any further on this project. But I did manage to clean up my DMDPwm library and I have posted it on the Freetronics forum board. (If anyone here is interested in it let me know)

You asked in another thread about the setup of the row data/latch and the column data/latch. The best I can off on that would be to tell you to have a look at my fritzing image on the previous page. It shows the connections I was using for my 16x16 colour matrix. But simply there are two chains of shift registers. One for column control and one for row control. Once you have them setup It should all work.

Once again long time between posts.....but we have progress.
I have obtained a USB to Serial adapter and have now tested connecting the TX of that to the RX of 2 Arduino's. This is what I have been banking on the whole time and it worked perfectly. Each Arduino waits for an assigned start byte and then acts on the bytes following the start byte. It then counts the bytes until it has a full module worth and displays them. The next start byte is received by the next Arduino and the sequence is repeated. This is how I plan to scale up to having 4 Arduino's working together, each controlling 2 DMD's. Thereby giving me a 64x64 resolution LED video screen.

I have ordered more panels (10 infact) and I am waiting for them to be delivered. Once I have them I will be attempting to scale up to full size.

In the mean time have a look at the 2 videos below. I used my camera phone and it is not great at capturing the LED's as they are quite bright. So I apologize for the image quality.

Hopefully in the coming weeks I will be able to show you the 64x64 in action.....I hope you like the colour GREEN!!!

Wow cool. All you need now is 1920 x 1080 leds with little space between them and you have a lovely wall sized hd display.
Maybe you can tap into free to air MPEG DBV feeds.
Imagine the power consumption though!

I just read your thread, im looking to apply shiftpwm to a 4x4x4 RGB cube (ShiftPWM3D ?)

I havnt been able to watch any of the videos or pictures, It says:
Sorry
There was an error encountered while loading this
video.

Have you posted the pix/video somewhere else? I would still like to see them :smiley:

Hi Hippynerd,

Not sure what is happening. I just tested the videos (here at work) and the ones in the thread do seem to be working. Unfortunately I don't have the videos hosted anywhere else. You could see if they work direct on the vimeo site (Rick). Everything in that profile is for this project.

I have thought of using ShiftMatrixPWM to do a cube. In 4x4x4 I would do 4x4 as a direct wire up. (4x4=16, 16x3=48 LED's) So I would use 6 SR's and then have 1 SR (That would have controlled the ROW) to control the Plane. This should be well within the range that ShiftMatrixPWM can do (only 192 individual LED's). I am not sure if ShiftMatrixPWM will work correctly this way but for the life of me I can't see any reason it would not? I would certainly ask in the ShiftPWM thread and see if anyone else has actually used this for a cube.

Thank you, I can see the videos on vimeo. i've tried on 2 computers here, and they both give me the same error message.
That is exactly how I had originally planned the RGB 4x4x4 cube, I even soldered up a stack of 7 shift registers all nicely piggybacked, but after experimenting with 8RGB LEDs and 3 shift registers, I decided to do 6 shift registers for the RGB, and I ended up using a ULN2003 to control the anodes (which I had thought were cathodes, based on their markings).

I soldered it up, but its not working right. I tried a few things, but have decided to put it down while I work on a charliecube design (just leds, wires and a nano).

Curious how this project went?
Anyone who can answer?

Mark Joseph Pongasi
Dynamo Led Display