HacroCam - recalculate to greyscale before sending (processing example)

HI,

I did have the same idea yesterday to get a 4bit gray-scale image.

Another idea is to send an image progressively. Suppose you have an image of 8x8 pixels. Then you send the pixels in this order:
// layer 0 lowest resolution (stepsize n/2)
[0,0]
[4,0]
[0,4]
[4,4]

// layer 1 (stepsize n/4 but not those already send)
[2,0], [6,0]
[0,2], [2,2], [4,2], [6,2]
[2,4], [6,4],
[0,6], [2,6], [4,6], [6,6]

// layer 2 (stepsize n/8 but not those already send => which are all remaining pixels for an 8x8 bitmap.
[1,0], [3,0], 5,0], [7,0]
etc

There advantage of progressive images is that you get an idea of the overall picture and it refines gradually.
It also allows to stop transmitting when the resolution is high enough.

Besides the proposed schema, there are many more schemas. The best known is the alternating lines.
First send all the even lines and than all odd lines.

Another speed up of transmission could be run length encoding. This is especially useful if you go to the 4 bit modes.

A byte could become a {run length; colour } tuple, with a run length of 1..15 and a 4 bit colour.
If you have 6 bytes of a certain colour (e.g. colour 9) you send { 6, 9 } or as byte B0110 1001

Hi,

send an image progressively is a good idea. Unfortunately in my case it will not work for two reasons:

The image gets streamed from the camera to the Arduino via TWI/I2C though a 32byte buffer (so 16 pixels).
Only the buffer can be reworked. That's because auf the limited RAM available. The image in total would not fit into the RAM.
I could download the image pixel by pixel from the cam. But this would slow things down again an negate the compression work done.

At the moment in Processing the picture doesn’t get rendered during the download. I tried to change that (to see the pixels arriving) but didn’t succeed. As I am one of the many victims of the “new topic button now showing in the Processing forum” bug, I can’t even post to get help there. ?

The run length encoding is interesting too. But I would lose half of the space for pixels. Maybe it would be possible to mix 4bit grayscale and run length encoding with some signaling byte. But as I can only process 16 pixels at a time, I am not sure if this in reality would improve speed. Taking into account the lost information space and the calculation time.
I looked at other compression methods. But they are all too heavy for an Arduino.

I also looked at other color schemes. 3-3-2 RGB looked nice too. But as far as I read the shading depth is more important for the human brain than the color. (especially for me, as I am partially color blind XD )

If you have more ideas – please through them in. Maybe we get it to stream a video over the 115k baud. ]:smiley:

Thanks
Robert

I tried the 4bit grayscale. The saved trasmission time gets eaten up by the calculation time. It may still be worth the effor, as it will help on speeds slower than 115k baud.

I think I got close but don't get a correct picture. I think the error is on the decoding side. But I don't get it. Maybe if I look at it later again.

(Still l and one char variables in, as it's the old code. The newly added variable names are better. :-))

encoding:

//*pbuf is the pointer to the 32 byte RGB pixel buffer

           void rgbTo4Bit (byte *pbuf, int bufflength){
		for (unsigned int l= 0; l < bufflength/4; ++l){
			int msbLeft = pbuf[4*l] & 0xff;
			int lsbLeft = pbuf[4*l+1] & 0xff;
			int rLeft = (lsbLeft >> 3) & 0x1F;
			int gLeft = ((lsbLeft & 0x07) << 3) | ((msbLeft >> 5) & 0x07) ;
			int bLeft = (msbLeft & 0x1f);
			rLeft = rLeft << 2;
			gLeft = gLeft << 1;
			bLeft = bLeft << 2; 
			int grayLeft=(77 * rLeft + 150* gLeft + 29 * bLeft) >> 8;
			grayLeft = grayLeft >> 4;
			grayLeft = grayLeft << 4;
			
			
			int msbRight = pbuf[4*l+2] & 0xff;
			int lsbRight = pbuf[4*l+3] & 0xff;
			int rRight = (lsbRight >> 3) & 0x1F;
			int gRight = ((lsbRight & 0x07) << 3) | ((msbRight >> 5) & 0x07) ;
			int bRight = (msbRight & 0x1f);
			rRight = rRight << 2;
			gRight = gRight << 1;
			bRight = bRight << 2; 
			int grayRight=(77 * rRight + 150* gRight + 29 * bRight) >> 8;
			grayRight= grayRight >> 4;
			
			pbuf[l]=(grayLeft | grayRight);
		} 
	}

decoding (in Processing):

for (int row = imageh-1; row >= 0; --row)
{
  for (int col = 0; col < imagew; col=col+2)
  {
    int loc = imagew*row+col;
          int LeftPixel= packedbuf[loc] >>> 4;
          LeftPixel= LeftPixel << 4;
          int RightPixel= packedbuf[loc] << 4;
          img.pixels[loc] = color(LeftPixel,LeftPixel, LeftPixel);
          img.pixels[loc+1] = color(RightPixel, RightPixel, RightPixel);
    }
  }

It loos like this: 4bit.jpg Picture
Must be something wrong with the col and row count I think,

Robert

decoding:

for (int row = imageh-1; row >= 0; --row)
{
  for (int col = 0; col < imagew; col += 2)
  {
    int loc = imagew*row + col;
    int LeftPixel = packedbuf[loc] & 0XF0;  // mask the higher 4 bits is enough
    int RightPixel = packedbuf[loc] << 4;

    img.pixels[loc] = color(LeftPixel,LeftPixel, LeftPixel);
    img.pixels[loc+1] = color(RightPixel, RightPixel, RightPixel);
  }
}

For encoding the 4 bit you can reuse your rgb2gray() because that almost delivers the right thing

somewhere in the code just add one line - in the download code where you patched the rgb buffer to gray

rgb2gray(buffer, length);
gray2doublegray(buffer, length/2); // adjust params

void gray2doublegray(byte *buf, int len)
{
  int pos = 0;
  for (int i = 0; i < len; i += 2)
  {
    int packed = buf[i] & 0xF0;
    packed += (buf[i+1] & 0xF0) >> 4;
    buf[pos++] = packed;
  }
}

get the idea? give it a try

Reusing the grb2gray function is indeed much more elegant. Same goes for the masking, which should also be more efficient.
I still have a problem with the decoding/placing the pixels (I think that's wher the problem is.). I found one bug and it looks better now. But it's still not correct.

I started a thread in the Processing forum about this specific topic.

Robert

Reusing the code makes it more modular and you could make one download function that depending on the parameter from processing it returns either colour, gray or doublegray

Hi robtillaart,

got the 4bit part now working too.

encoding:

int grayLeft=pbuf[A] & 0xF0;
int grayRight=(pbuf[A+1] & 0xF0) >> 4;
pbuf[pos]=(grayLeft | grayRight);

decoding:

int LeftPixel = packedbuf[loc/2] & 0xF0;
int RightPixel = (0x0F & packedbuf[loc/2]) << 4;
img.pixels[loc] = color(LeftPixel, LeftPixel, LeftPixel);
img.pixels[loc+1] = color(RightPixel, RightPixel, RightPixel);

Regarding to 5/6 bit to 8 bit transcoding:
until now I used:

r = r << 2;
g = g << 1;
b = b << 2;

you suggested (gived the black areas):

r = r << 3;
g = g << 2;
b = b << 3;

The processing code uses:

r = (r << 3)|(r >>> 2);
g = (g << 2)|(g >>> 4);
b = (b << 3)|(b >>> 2);

I now use:

r = (r << 2)|(r >> 1);
g = (g << 1)|(g >> 3);
b = (b << 2)|(b >> 1);

This gives quite good results. I still don't understand why I have to shift one bit less than even the Processing code does.

At least it works.

Robert

How about the performance?
And quality of the images (good enough)?

The performance is good. Bit below 8bit the calculation time eats the time gain in transmition. At least with 115k baud. With less transmition speed the 4 and 2 bit options become interesting.
The quality is ok, but not good.
I think the 8 bit grayscale is only 7 bit (so only contains 127 color shades). If I go down further, so 4 or 2 bit I need to shift different at the initial RGB to grayscale convertion.
Otherwise I get black blocks.

I now have a rgb28bit function where I shift

r = (r << 2)|(r >> 1);
g = (g << 1)|(g >> 3);
b = (b << 2)|(b >> 1);

and a rgb28bit_low function (to be used for rgb to 4 or 2 bit)

r = (r << 3)|(r >> 2);
g = (g << 2)|(g >> 4);
b = (b << 3)|(b >> 2);

All a bit strange.

YOu can speed up the code by going though the code you changed and look where you can replace 16 bits ints with 8 bits ones. Remove temporary vars.
As the same math is done for every pixel this adds up. Furthermore you can do more in place

e.g.

int grayLeft=pbuf[A] & 0xF0;
int grayRight=(pbuf[A+1] & 0xF0) >> 4;
pbuf[pos]=(grayLeft | grayRight);

void gray2doublegray(byte *buf, byte len)
{
  byte pos = 0;
  for (byte i = 0; i < len; i += 2)
  {
    buf[pos++] = (buf[i] & 0xF0) |( (buf[i+1] & 0xF0) >> 4);
  }
}

Thanks for the advice. I'll go through the code an see what can be changed.

I hope the Hcrocam guys are able to solve the shifting puzzle. I would like to have it as it is supposed to be.
They were interested int eh work. So they may come by after their vacation.

img.pixels[loc] = color(LeftPixel,LeftPixel, LeftPixel);

does img.pixels[loc] = LeftPixel; work? [processing]

Hi,

no, this doesn't work. It doesn't give a picture at all.
The image is defines as:

PImage img = createImage(imagew, imageh, RGB);

So I think it expects three values. Possible formats are RGB, ARGB and APLHA.

Did the int -> byte affect the performance?

At which point do you mean? The convertion from grayscale to color() data type in processing?

See reply #26 - HacroCam - recalculate to greyscale before sending (processing example) - #27 by robtillaart - Sensors - Arduino Forum -

No, it doesn't make a difference. But I use it anyway now. It saves some memory.

Sorry if I'm breaking the necro rule with this reply - I couldn't find anything explicit saying what the rule is - but I'm using a HacroCam myself and have a suggestion for using greyscale.

If you look at hacromatic.com, they talk about "switch[ing] the camera over into YUV mode and then just read the Y byte" "so we can store a pixel in a single byte" rather than going through the kinds of hoops that you went through above. The downsides are that 1) you need to change the HacroCam's firmware to do this, requiring a PDI programmer, and 2) that you won't have the option of enabling and disabling it at will. The upside is that every pixel is only 8 bits, speeding everything else up.

I'm using the tutorial on that page myself, though I'm having different trouble that I'll be posting in a new thread.

Nice to see that there is at least one other user with a Hacrocam! :slight_smile:

Thanks a lot for the information. I don't have a programmer - so nothing I could test. But thanks anyway.