ShiftMatrix, Dreaming Big Video Display

I have been doing some tests with the ShiftMatrix library (http://arduino.cc/forum/index.php/topic,66988.0.html) from ElcoJacobs. The latest version seems to be stable and good for a purpose I have been working toward. I have been working on making a LED Video Display for a christmas display. Originally I was basing it on the Peggy 2.0 design from Evil Mad Scientists. This allowed 25x25 monochrome. I even managed to make a processing sketch to feed it video from a windows computer (I don't have Mac). This was good but I wanted bigger/better. Hence this thread.

I have Elco's library working, giving me 16x48 LED's with 32 levels of brightness. This is currently Monochrome. I want to go RGB. So 48/3=16 giving me 16x16 RGB. But I want bigger than this. I have seen many people wanting to accomplish big things with the Arduino and the comments that go with them tend to be similar in that they say "Make it Modular". So if I make a 16x16 RGB section with LED drivers and accompaning Arduino as one module, with 4 modules I could have 32x32 RGB Matrix with 32 levels of brightness.

This is all theory at this stage, I only have 1 breadboarded prototype. My next question is how to get the frame data to the devices? And can I do it at say 15fps. At 32x32 thats 1024 RGB LED's. Thats 3072 individual LED brightness levels. Sending each LED's level as a byte is 3072 x 8= 24576 bits. Thats for 1 Frame, 24576 x 15 = 368640 bps. Thats just the data, not sure what the overhead would be?

My first thought was to use I2C. This would mean using a seperate Arduino as a usb -> I2C master. Or I could use serial, linking all the Rx pins togther. I would need start and stop headers for each module and all data would be sent to all micros but they would only act on the data with their header.

I suppose I am asking if all this is feasable or am I just aiming too high?
Also I am sorry for the cross post about speed/interfacing. I will keep it all here as the project is actually about a LED matrix.

First thing I'd ask myself is "do I really need 15FPS". :slight_smile: Work from requirements to solution rather than vise-versa, otherwise you'll waste a bunch of time and drive yourself nuts in the process.

Do I "Need" it. Probably not, it is a want. The desire was to do video. It is what I would like to aim for. As I said though, I am not sure if it can be done. If the speeds are not attainable then I will have to scale back. My next step is to see if I can adapt my Video streaming Peggy code to stream the data needed for the ShiftMatrix library. I don't think this will be too hard. And I will see if I can get a receive sketch running on the Arduino. Start with one 16x16 set and go from there. I will post back with any results. If anyone else has anything to add please do. :slight_smile:

Ok. I have modified the Proccessing sketch I had for sending video to the Peggy. Currently is now downscalling video to 16x16 monochrome, building an array of bytes and then sending that array down the serial connection. As far as I can tell I have that bit working fine.

On the arduino end I have had a bit of trouble with reading the serial stream. I am at work currently but I think I have figured out what I was doing wrong. Does this sort of code look right?

char byteToRead;

void setup()
{
	Serial.begin(115200);
	int outPWM = 0;
}

void loop()
{
	if(Serial.available()){
		byteToRead = Serial.read();
		If(byteToRead == 127){ //is it "~" start the frame
			outPWM = 0;
		}
		else{
			ShiftMatrixPWM.m_PWMValues[outPWM] = byteToRead;
			outPWM++;
		}
	}
}

I know this isn't the full code. I just want to know if I am getting the serial read to happen right. I am using "~" as the start frame. Then there will be 256 bytes of PWM data. Then it repeats for each frame of the video. ShiftMatrixPWM.m_PWMValues[] is what the ShiftMatrix Library uses to run the PWM.

Ok. I got it working. Had to wrap my head around receiving serial data but all good. And simple once you understand. So setup is exactly as in Elco's example (settings to suit my matrix config) I have added two variables.
byte byteToRead;
int outPWM = 0;
Before the Setup function. The Setup function is unchanged from the example. And this is what I have for the Loop function:

void loop()
{    
  // Print information about the interrupt frequency, duration and load on your program
  //ShiftMatrixPWM.SetAll(0);
  //ShiftMatrixPWM.PrintInterruptLoad();
  
  if(Serial.available()){
    byteToRead = Serial.read();
    
    if (byteToRead == '~'){ //is it "~" start the frame
      outPWM = 0;
    }
    else{
      ShiftMatrixPWM.m_PWMValues[outPWM] = byteToRead ;
      outPWM++;
    }
  }
}

Its currently setup for 16x16 mono but it is working at way below capacity. Image is good and clear (for 16x16) and is updating very fast. Currently have it set for 31 levels of brightness at 15fps. I am very pleased that I got it to work. If anyone is interested I will post the code of how I got the video downscaled and sent serially. It is a Processing sketch using GSVideo Library.

I think I will have to remake my matrix, its in a 24x24 arrangement at the moment. I will make it into a 16x48 setup and see how well it works with the full 768 LED's running. I will have to modify my conversion sketch too. Lots to do. But it is working!

Hi, i'm quite interested in updates of your project, mainly because i have a similar project ongoing. What i have is a 8x8 RGB matrix, with a colorduino and currently i am driving it at 15-20fps with 253 brightness levels. My first test is a program in C# which captures realtime screenshots and feeds them to an arduino. that way i can test a whole bunch of things like video, color test patterns etc without the need to modify the code since all i'm doing is streaming a portion of the desktop.

this is my thread if you want to read more. i can also provide you with some more code...

i am really getting awesome results with video

I will follow your thread to see where this is going. :slight_smile:

Elco asked for a video of it in action on the shiftPWM thread, so here it is again on my thread:
This is the Bad Apple video scaled to 16x16 monochrome at 15fps/32 brightness levels 50Hz.
The flickering is a camera artifact and is not present looking at the display.

Hey Zuluriney, I thought you might see this. :slight_smile: I am using processing to do my scaling and sending. I am not confident enough with C yet and there was a video library with examples that I could hack. So far I am really happy with the output. I will be checking out your thread too :slight_smile:

nice! very good so far, but what is the horizontal gap about? did everything not fit behind the matrices?

The gap is due to this being on some crappy breadboards and so the 8x8 matrix's don't match up to each other. But as a test bed I am making some good progress. I am just now completely re-writing the processing sketch. It's now much clearer, to my mind at least, as to what is going on. And I am commenting it as fully as I can so others can follow along.

I hope to have it ready to post up here sometime soon. The only issue with it that I can see is that I wrote it for a specific set of parameters. For example it is coded for a square matrix. It also assumes the input video to be 4:3 ratio. I have included comments to show where the code makes these assumptions and they can be coded to change. The other thing is it is currently set to output as a mono display. I don't have a color matrix yet but I have them on order. When I receive them I will be coding it for color.

As a test I pushed it to 90fps at 16x16 mono with 32 brightness levels and it had no glitches. I was impressed.

may i ask about how you construct the packets to send via serial? any buffering issues?

depending on how much RAM on the arduino is used by shiftPWM and your code, there is a way to increase buffer size, just FYI.

 if (byteToRead == '~'){ //is it "~" start the frame
      outPWM = 0;
    }
    else{
      ShiftMatrixPWM.m_PWMValues[outPWM] = byteToRead ;
      outPWM++;
    }

So what happens when one or more of the 256 PWM values is the same as the ASCII value of '~'?

I am sending one start byte. Then I am sending the whole frame array in one command. I am not currently using an end byte, that may change in future.

I have not seen any buffering issues. I'm not an expert but from what I can tell the micro is running much faster than the baud rate of the serial. When a byte comes into the serial buffer my routine runs very quickly and finishes well before another byte even gets to the buffer. I actually made that mistake when I first tried to make this. I was detecting when the first byte arrived and then telling it to get all the frame bytes in a for loop. This didn't work. I did some debugging and finally realized that I was getting the first byte but all the other Serial.reads were not getting anything. Then it hit me. The micro is running at 16Mhz or 16 000 000hz where the serial data is coming in at (in my tests) 115 200hz. That is 138 times slower than the micro. So instead of a for loop to fill the array I changed to a counter being increased (outPWM). So now it detects a byte has arrived. Checks if it is the start byte. If yes it resets the counter and then exits the Serial.available if statement. If its not the start bit, it puts the byte into the library's PWM array, using the counter to keep track of which array position should be filled. Then again exits the Serial.available if statement.

I have no error checking or qualifying. It expects to be sent the correct data in the right format. When I test going modular with this I will be changing this slightly. I will probably put another if statement in to check if the counter is above the maximum number of pixels. Therefore it will do nothing with the byte that is read.

About the start byte. The values I am using at the moment for PWM are 0-31. '~' is value 126. So it is not possible for a frame value to be the same. If you wanted to have 0-255 I would suggest modifying a 255 value to be 254 and then use 255 as a start byte. But thats just one way you could do it. :slight_smile:

I hope the above all makes sense. :slight_smile:

I am waiting for my RGB matrix order to arrive and have been doing some more testing. Using the ShiftMatrix library you can actually get control of up to 1024 individual LED's. Out of the two configurations 64*16 is the best as it allows 32 levels of brightness. And with 16 rows the LED's still look nice and bright.

I am now wondering what would look better? By making these modular as I was looking at doing, I could make a 64x64 Mono color matrix with just 4 Arduino's and quite a number of shift registers. Or do I continue and go color with a 32x32 RGB matrix?

What are your opinions, "Color over more dots" or "more dots over color"? Which do you think is more important when looking at a down scaled video? The final matrix is meant to be an outside display as part of a Christmas feature. So it will be big and viewed from a distance. This is the hope at least. I am starting to think I am not going to have it together in time. :fearful:

I would go for color over resolution but as you need it for christmas i'd say go with mono red or green as it fits the theme. Also i would do some testing with the LED distance and the distance you want to look at it from.

Well it has taken a long time for my colour matrix parts to arrive but I have it up and running. Initial tests were very disappointing. Colour was extremely washed out and very hard to make anything out. I ran test patterns which showed that all the LED's were functioning as expected in the matrix. So I did some reading about outputing colours on an RGB LED and learnt some more about how the human eye responds to light. Long and the short of it was that the linear application of colour from the Processing script was messing things up (i.e. the 0-255 levels). I was simply dividing by 8 to get 0-31 brightness levels. This did not translate well on the Matrix and looked TERRIBLE! After some googling I found this "INT(31^(LEVEL/31)+0.5)", it takes the original 0-31 brightness value and translates those levels into better levels for our eyes to identify. Using this has dramictally improved the picture on the Matrix. Colours look much clearer and closer to what I would expect to see.

Unfortunately all this waiting for parts (and other outside forces) have delayed the project and I don't think I will be able to complete it in time for Christmas. I am disappointed but I will be continuing on, in the blind hope I might get something ready. After all my tests I am going to go with Mono Green at 64x64 resolution with 32 brightness levels. This will be using four 16x64 modules. I see much soldering in my future. Mono colour seems to be easier to build and cheaper to source. Even at 16x16 in color I found the picture very hard to make out. And the colour version I was planning was only going to be 32x32. I am hoping that better resolution will make up for the lack of colour.

Let me know what you guys think.

would be nice to see some pictures of the setup ^^

yeah I'd love to see a video of the color, even if you do think it's not that great yet, just to see where you are. very impressed. This is way above my head at the moment.

Alright, video it is!

I am not sure if you will be able to make it all out as I filmed it close up with a piece of paper as a diffuser. Here is the way it looked when I first connected it up. This is using the same scaling that is in the Bad Apple video (The all red one). It is simply reading the RGB values as 0-255 and dividing by 8 to get 0-31.

As I posted before I was very disappointed with the above result. Everything was washed out and it was hard to make out what was being drawn to the matrix. I did a lot of research and changed the way it output values. It still gave me value between 0 and 31 but it weighted it differently. Its difficult to explain but simply our eyes are better at discerning the difference between PWM 1 and PWM 2. But now so good at seeing the difference between PWM 30 and PWM 31(this is using my 32 level system, this also holds true on 0-255 PWM systems). So with adjustments made the results look like this.

This looked a lot better. It should be a lot easier to make out the red for example in the video above than the one above it.

I know I haven't put up the processing code yet. I have done HEAPS of tweaking to it over the last month. Hopefully I am close to posting it soon. But here is a quick video of what I show on the screen. I allowed me to get things setup correctly even without having the Arduino and the matrix hooked up.

In the top left is the original frames playing in their native size. Beside that is the down-scaled version, in this case it is 16x16 then doubled in size so you can see it. Below those is a representation of what should be output to the RGB matrix display.

As an aside, when I post up the processing code would you prefer it for single colour or RGB, as they are a bit different? Or would you prefer to see both?

I am now ready to move out of testing and to start building the large scale matrix itself. As I said before I am planning to go 64x64 mono. But I am scared that it will be huge(I was planning to use Peg Board as a pre-drilled base board) as it is looking like it will be about 2m x 2m in size. I might chicken out a bit and bring it back to 48x48 mono.....just not sure. Pricing lots of LED's now. I don't suppose anyone knows a good source for 4500 LED's at a cheap price, for delivery to Australia. Other than Ebay that is.
So any other comments or queries?

That looks really good. Nice work. I am not sure about large lots of LEDs other than the usual suspect(ebay). That's going to be a hell of a build though. Get some friends, and make a weekend of it :wink:

For anyone who is interested I have attached a Fritzing image of how the RGB Matrix in the above posts is actually setup. I have recieved a few queries about how the shift registers are connected to the Arduino, so I hope this helps people. It shows that there are two sets of DATA, CLK and Latch that come from the Arduino.

As I have said in other posts the IC's I have used are MBI5026 (16ch, constant current, serial in Parrallel out) and the PNP transistors were rated up to 1.5amp. Capacitors are 0.1uf Ceramic type. Resistors are for IREF of the IC's (currently I am actually using a trimpot on each IC)

I have omitted the connections from the IC's to the columns of the Matrix. Also missing are the wires from the Row IC to the PNP transistors as well as the connections from the transistors to the Rows. This is simply because there would be A LOT of wire and if would acutally make it more difficult to follow what is going on and how it is setup. (about 90 connections omitted, not including the connections not shown to form the 16x16 matrix) If you are going to attempt something like this on breadboard be prepared to be sitting there wiring it up for a LONG time!