Pages: 1 [2] 3   Go Down
Author Topic: HacroCam - recalculate to greyscale before sending (processing example)  (Read 3832 times)
0 Members and 1 Guest are viewing this topic.
Offline Offline
Full Member
***
Karma: 1
Posts: 152
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

@Hacrocam users

Attached you find an updated arduino library, the arduhub sketch and an updated processing example for the Hacrocam.
The update allows to convert the image to 8bit grayscale before submitting it via serial link. This halves the time required for the transfer. Great for low baud rate serial links.
In addition you can take more than one picture without restarting the sketch. Press w to take a picture.

The processing example has an added user definable variable g_grayscaleImage to enable/disable grayscale convertion.

Many thanks @robtillaart who solved the hard part. I just put together the puzzle pieces and extended the library and sketches.

Trying to extend it with a 4bit grayscale option now.

Robert

* Hacrocam_8bit_option.zip (19.47 KB - downloaded 19 times.)
Logged

Global Moderator
Netherlands
Offline Offline
Shannon Member
*****
Karma: 216
Posts: 13676
In theory there is no difference between theory and practice, however in practice there are many...
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

HI,

I did have the same idea yesterday to get a 4bit gray-scale image.

Another idea is to send an image progressively. Suppose you have an image of 8x8 pixels. Then you send the pixels in this order:
// layer 0 lowest resolution (stepsize n/2)
[0,0]
[4,0]
[0,4]
[4,4]

// layer 1  (stepsize n/4 but not those already send)
[2,0], [6,0]
[0,2], [2,2], [4,2], [6,2]
[2,4], [6,4],
[0,6], [2,6], [4,6], [6,6]

// layer 2 (stepsize n/8 but not those already send => which are all remaining pixels for an 8x8 bitmap.
[1,0], [3,0], 5,0], [7,0]
etc

There advantage of progressive images is that you get an idea of the overall picture and it refines gradually.
It also allows to stop transmitting when the resolution is high enough.

Besides the proposed schema, there are many more schemas. The best known is the alternating lines.
First send all the even lines and than all odd lines.

Another speed up of transmission could be run length encoding. This is especially useful if you go to the 4 bit modes.

A byte could become a {run length; colour } tuple, with a run length of 1..15 and a 4 bit colour.
If you have 6 bytes of a certain colour (e.g. colour 9) you send { 6, 9 } or as byte B0110 1001

Logged

Rob Tillaart

Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -
(Please do not PM for private consultancy)

Offline Offline
Full Member
***
Karma: 1
Posts: 152
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Hi,

send an image progressively is a good idea. Unfortunately in my case it will not work for two reasons:

1)
The image gets streamed from the camera to the Arduino via TWI/I2C though a 32byte buffer (so 16 pixels).
Only the buffer can be reworked. That's because auf the limited RAM available. The image in total would not fit into the RAM.
I could download the image pixel by pixel from the cam. But this would slow things down again an negate the compression work done.

2)
At the moment in Processing the picture doesn’t get rendered during the download. I tried to change that (to see the pixels arriving) but didn’t succeed. As I am one of the many victims of the “new topic button now showing in the Processing forum” bug, I can’t even post to get help there. 


The run length encoding is interesting too. But I would lose half of the space for pixels. Maybe it would be possible to mix 4bit grayscale and run length encoding with some signaling byte. But as I can only process 16 pixels at a time, I am not sure if this in reality would improve speed. Taking into account the lost information space and the calculation time.
I looked at other compression methods. But they are all too heavy for an Arduino.

I also looked at other color schemes. 3-3-2 RGB looked nice too. But as far as I read the shading depth is more important for the human brain than the color. (especially for me, as I am partially color blind  smiley-lol )

If you have more ideas – please through them in. Maybe we get it to stream a video over the 115k baud.  smiley-twist

Thanks
Robert
Logged

Offline Offline
Full Member
***
Karma: 1
Posts: 152
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

I tried the 4bit grayscale. The saved trasmission time gets eaten up by the calculation time. It may still be worth the effor, as it will help on speeds slower than 115k baud.

I think I got close but don't get a correct picture. I think the error is on the decoding side. But I don't get it. Maybe if I look at it later again.

(Still l and one char variables in, as it's the old code. The newly added variable names are better. :-))

encoding:
Code:
//*pbuf is the pointer to the 32 byte RGB pixel buffer

           void rgbTo4Bit (byte *pbuf, int bufflength){
for (unsigned int l= 0; l < bufflength/4; ++l){
int msbLeft = pbuf[4*l] & 0xff;
int lsbLeft = pbuf[4*l+1] & 0xff;
int rLeft = (lsbLeft >> 3) & 0x1F;
int gLeft = ((lsbLeft & 0x07) << 3) | ((msbLeft >> 5) & 0x07) ;
int bLeft = (msbLeft & 0x1f);
rLeft = rLeft << 2;
gLeft = gLeft << 1;
bLeft = bLeft << 2;
int grayLeft=(77 * rLeft + 150* gLeft + 29 * bLeft) >> 8;
grayLeft = grayLeft >> 4;
grayLeft = grayLeft << 4;


int msbRight = pbuf[4*l+2] & 0xff;
int lsbRight = pbuf[4*l+3] & 0xff;
int rRight = (lsbRight >> 3) & 0x1F;
int gRight = ((lsbRight & 0x07) << 3) | ((msbRight >> 5) & 0x07) ;
int bRight = (msbRight & 0x1f);
rRight = rRight << 2;
gRight = gRight << 1;
bRight = bRight << 2;
int grayRight=(77 * rRight + 150* gRight + 29 * bRight) >> 8;
grayRight= grayRight >> 4;

pbuf[l]=(grayLeft | grayRight);
}
}

decoding (in Processing):
Code:
for (int row = imageh-1; row >= 0; --row)
{
  for (int col = 0; col < imagew; col=col+2)
  {
    int loc = imagew*row+col;
          int LeftPixel= packedbuf[loc] >>> 4;
          LeftPixel= LeftPixel << 4;
          int RightPixel= packedbuf[loc] << 4;
          img.pixels[loc] = color(LeftPixel,LeftPixel, LeftPixel);
          img.pixels[loc+1] = color(RightPixel, RightPixel, RightPixel);
    }
  }

It loos like this: http://picturepush.com/public/11920881
Must be something wrong with the col and row count I think,

Robert
Logged

Global Moderator
Netherlands
Offline Offline
Shannon Member
*****
Karma: 216
Posts: 13676
In theory there is no difference between theory and practice, however in practice there are many...
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset


decoding:
Code:
for (int row = imageh-1; row >= 0; --row)
{
  for (int col = 0; col < imagew; col += 2)
  {
    int loc = imagew*row + col;
    int LeftPixel = packedbuf[loc] & 0XF0;  // mask the higher 4 bits is enough
    int RightPixel = packedbuf[loc] << 4;

    img.pixels[loc] = color(LeftPixel,LeftPixel, LeftPixel);
    img.pixels[loc+1] = color(RightPixel, RightPixel, RightPixel);
  }
}
Logged

Rob Tillaart

Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -
(Please do not PM for private consultancy)

Global Moderator
Netherlands
Offline Offline
Shannon Member
*****
Karma: 216
Posts: 13676
In theory there is no difference between theory and practice, however in practice there are many...
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

For encoding the 4 bit you can reuse your rgb2gray() because that almost delivers the right thing

somewhere in the code just add one line - in the download code where you patched the rgb buffer to gray

   rgb2gray(buffer, length);
   gray2doublegray(buffer, length/2);  // adjust params

Code:
void gray2doublegray(byte *buf, int len)
{
  int pos = 0;
  for (int i = 0; i < len; i += 2)
  {
    int packed = buf[i] & 0xF0;
    packed += (buf[i+1] & 0xF0) >> 4;
    buf[pos++] = packed;
  }
}
   
get the idea? give it a try
« Last Edit: January 12, 2013, 08:42:24 am by robtillaart » Logged

Rob Tillaart

Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -
(Please do not PM for private consultancy)

Offline Offline
Full Member
***
Karma: 1
Posts: 152
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Reusing the grb2gray function is indeed much more elegant. Same goes for the masking, which should also be more efficient.
I still have a problem with the decoding/placing the pixels (I think that's wher the problem is.). I found one bug and it looks better now. But it's still not correct.

I started a thread in the Processing forum about this specific topic.
https://forum.processing.org/topic/writing-pixels-to-a-pimage-positioning-pixels-right

Robert
Logged

Global Moderator
Netherlands
Offline Offline
Shannon Member
*****
Karma: 216
Posts: 13676
In theory there is no difference between theory and practice, however in practice there are many...
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Reusing the code makes it more modular and you could make one download function that depending on the parameter from processing it returns either colour, gray or doublegray

Logged

Rob Tillaart

Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -
(Please do not PM for private consultancy)

Offline Offline
Full Member
***
Karma: 1
Posts: 152
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Hi robtillaart,

got the 4bit part now working too.

encoding:
Code:
int grayLeft=pbuf[A] & 0xF0;
int grayRight=(pbuf[A+1] & 0xF0) >> 4;
pbuf[pos]=(grayLeft | grayRight);
decoding:
Code:
int LeftPixel = packedbuf[loc/2] & 0xF0;
int RightPixel = (0x0F & packedbuf[loc/2]) << 4;
img.pixels[loc] = color(LeftPixel, LeftPixel, LeftPixel);
img.pixels[loc+1] = color(RightPixel, RightPixel, RightPixel);


Regarding to 5/6 bit to 8 bit transcoding:
until now I used:
Code:
r = r << 2;
g = g << 1;
b = b << 2;
you suggested (gived the black areas):
Code:
r = r << 3;
g = g << 2;
b = b << 3;
The processing code uses:
Code:
r = (r << 3)|(r >>> 2);
g = (g << 2)|(g >>> 4);
b = (b << 3)|(b >>> 2);
I now use:
Code:
r = (r << 2)|(r >> 1);
g = (g << 1)|(g >> 3);
b = (b << 2)|(b >> 1);

This gives quite good results. I still don't understand why I have to shift one bit less than even the Processing code does.

At least it works.

Robert
« Last Edit: January 11, 2013, 04:45:52 am by robvoi » Logged

Global Moderator
Netherlands
Offline Offline
Shannon Member
*****
Karma: 216
Posts: 13676
In theory there is no difference between theory and practice, however in practice there are many...
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset


How about the performance?
And quality of the images (good enough)?
Logged

Rob Tillaart

Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -
(Please do not PM for private consultancy)

Offline Offline
Full Member
***
Karma: 1
Posts: 152
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

The performance is good. Bit below 8bit the calculation time eats the time gain in transmition. At least with 115k baud. With less transmition speed the 4 and 2 bit options become interesting.
The quality is ok, but not good.
I think the 8 bit grayscale is only 7 bit (so only contains 127 color shades). If I go down further, so 4 or 2 bit I need to shift different at the initial RGB to grayscale convertion.
Otherwise I get black blocks.

I now have a rgb28bit function where I shift
Code:
r = (r << 2)|(r >> 1);
g = (g << 1)|(g >> 3);
b = (b << 2)|(b >> 1);

and a rgb28bit_low function (to be used for rgb to 4 or 2 bit)
Code:
r = (r << 3)|(r >> 2);
g = (g << 2)|(g >> 4);
b = (b << 3)|(b >> 2);

All a bit strange.
Logged

Global Moderator
Netherlands
Offline Offline
Shannon Member
*****
Karma: 216
Posts: 13676
In theory there is no difference between theory and practice, however in practice there are many...
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset


YOu can speed up the code by going though the code you changed and look where you can replace 16 bits ints with 8 bits ones. Remove temporary vars.
As the same math is done for every pixel this adds up. Furthermore you can do more in place

e.g.
Code:
int grayLeft=pbuf[A] & 0xF0;
int grayRight=(pbuf[A+1] & 0xF0) >> 4;
pbuf[pos]=(grayLeft | grayRight);

void gray2doublegray(byte *buf, byte len)
{
  byte pos = 0;
  for (byte i = 0; i < len; i += 2)
  {
    buf[pos++] = (buf[i] & 0xF0) |( (buf[i+1] & 0xF0) >> 4);
  }
}
Logged

Rob Tillaart

Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -
(Please do not PM for private consultancy)

Offline Offline
Full Member
***
Karma: 1
Posts: 152
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Thanks for the advice. I'll go through the code an see what can be changed.

I hope the Hcrocam guys are able to solve the shifting puzzle. I would like to have it as it is supposed to be.
They were interested int eh work. So they may come by after their vacation.
Logged

Global Moderator
Netherlands
Offline Offline
Shannon Member
*****
Karma: 216
Posts: 13676
In theory there is no difference between theory and practice, however in practice there are many...
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Quote
    img.pixels[loc] = color(LeftPixel,LeftPixel, LeftPixel);

does   img.pixels[loc] = LeftPixel;   work? [processing]
Logged

Rob Tillaart

Nederlandse sectie - http://arduino.cc/forum/index.php/board,77.0.html -
(Please do not PM for private consultancy)

Offline Offline
Full Member
***
Karma: 1
Posts: 152
View Profile
 Bigger Bigger  Smaller Smaller  Reset Reset

Hi,

no, this doesn't work. It doesn't give a picture at all.
The image is defines as:
Code:
PImage img = createImage(imagew, imageh, RGB);
So I think it expects three values. Possible formats are RGB, ARGB and APLHA.
Logged

Pages: 1 [2] 3   Go Up
Jump to: