I have a camera connected to my Arduino board. It is now sending me JPEG data package. I am now confused as to what to do with these packages. How does JPEG imaging work? How can I convert these "data" into an actual picture/image?
Camera Data sheet:
Not with an Arduino - not enough speed, ram, etc...
I know. But I am sending the images over to the computer via USB and doing some image processing there. Is this still too slow? I only need to take one pic every 3 seconds.
dropsofjupiter89:
I know. But I am sending the images over to the computer via USB and doing some image processing there. Is this still too slow? I only need to take one pic every 3 seconds.
Well, if that's what you're doing, then on the PC there should already be a library in whatever language or system you are using to decode the jpeg into a framebuffer (or whatnot) for the image processing to take place on.
Depending on what you are planning to do, though, image processing the jpeg image may or may not be the best thing. What I mean by that is jpeg has artifacts due to the compression method it uses; there's no way around this, because it is a lossy compression scheme (can't get back data that isn't there to begin with). Depending on how heavy the compression is, it may effect your processing - but it depends on what you are doing, and whether you can live with the inaccuracy in the project.
Thanks for the info so far. Much appreciated.
I am using the camera to track the position of an object. The object is at least 10 inches from the camera, and it can move as far as 30 inches. I plan to put a small red sticker on the object and track the size of the sticker. The smaller the size, the further the object is. The larger the size, the closer the object is.
I just got the camera (with 160x128 resolution) to send jpeg data (i.e series of jpeg packages) to the arduino in less than 53 milliseconds. As I said before, I plan to send this data over to the computer via another serial comm.
I am still stick on what to do with this data. You said the program I am using should have this library? I am using "processing" (the arduino's counterpart). It does not have a jpeg library. I would really appreciate it if you could help me out more or point me to the right direction.
Also as a side question, is it possible to send the Camera's serial data directly to the computer without having to use the Arduino's serial port? I am asking because I am limited by Arduino's Rx buffer size ( I currently increased it to 1024 bytes, however, it if I could get access to a larger buffer size, then I could transfer the RAW picture instead of the JPEG picture [because processing RAW images would be much simpler than JPEG images]) ... don't worry about this question if its too confusing.
dropsofjupiter89:
Also as a side question, is it possible to send the Camera's serial data directly to the computer without having to use the Arduino's serial port? I am asking because I am limited by Arduino's Rx buffer size ( I currently increased it to 1024 bytes, however, it if I could get access to a larger buffer size, then I could transfer the RAW picture instead of the JPEG picture [because processing RAW images would be much simpler than JPEG images]) ... don't worry about this question if its too confusing.
The camera has an rs-232 output. Hook it up to a serial port on your computer, or if your computer doesn't have one, a USB to rs-232 adapter.
I think you'd be better off doing this. The arduino lacks the power to do the kind of image processing you want to do. It should be a piece of cake for a computer.
I am still stick on what to do with this data. You said the program I am using should have this library? I am using "processing" (the arduino's counterpart). It does not have a jpeg library. I would really appreciate it if you could help me out more or point me to the right direction.
A simple thing to do is take the arduino out of the picture and directly connect the cam to the computer using another USB toserial adapter operating on another com port. If the processing application has a "run" feature, you probably could have the image displayed by your default brouser.
I am using the camera to track the position of an object. The object is at least 10 inches from the camera, and it can move as far as 30 inches. I plan to put a small red sticker on the object and track the size of the sticker. The smaller the size, the further the object is. The larger the size, the closer the object is.
I just got the camera (with 160x128 resolution) to send jpeg data (i.e series of jpeg packages) to the arduino in less than 53 milliseconds. As I said before, I plan to send this data over to the computer via another serial comm.
How long does it take to send an uncompressed image? I bet a lot less than 3 seconds if a JPEG is 53 mSec.
For what you want it sounds like you need minimal processing -- look at each bit, and see if the color attribute is red. Count how many bits are red and keep track of where they are. If you read all the bits and then discard them this is well within the capabilities of the arduino.
Yeah, an uncompressed image takes more than 1 second (I don't have the exact value, I will check it out soon). However, I have made some good progress which I think is worth sharing with everyone else.
A bit of background info will be useful here.
The arduino nano has one rx/tx (serial) port. This is used via HardwareSerial.h
A newSoftSerial library allows you to communicate replicate the rx/tx functionality on any digital pin. This is used via (new)SoftwareSerial.h
If you modify the read() function in SoftwareSerial.cpp such that it calls write() function in HardwareSerial.cpp (instead of storing things in the serial buffer - which is limited to 64b,128b,256b, 512b...), you can transfer the serial data being received from the camera directly to the computer.
With a bit of help, I did this and it is now working -- that is, the camera directly feeds its serial data into the computer.
The only problem I had before was that the camera serial data transfer was TOO fast - and this eventually caused a serial buffer overflow. Now, instead of writing to the arduino serial buffer, I immediately transfer the data to the computer using the rx pin.
I know that sounds confusing. The only reason I am writing this is so anyone else having similar problems can have reassurance that there is in fact a solution.
To summarize, you can transfer serial data from any device to the computer via the Arduino board without having to be limited by the arduino's serial buffer, memory capabilities or clock speed (unstrike if you are using 8Mhz or higher).
If you modify the read() function in SoftwareSerial.cpp such that it calls write() function in HardwareSerial.cpp (instead of storing things in the serial buffer - which is limited to 64b,128b,256b, 512b...), you can transfer the serial data being received from the camera directly to the computer.
Can you post the modified part of the SoftwareSerial.cpp code?
Its not complicated, but I have little pieces of code all over the place. I will post the files below after a few comments/explanation.
1] HardwareSerial.h needs a new constructor, which is simply a dummy constructor. We need it because we need to create a HardwareSerial object in SoftwareSerial. I included both the HardwareSerial.h and .cpp files below.
2] SoftwareSerial has a function setHardwareSerial(). setHardwareSerial() is to be called in setup() of your arduino code as follows:
#include <SoftwareSerial.h>
#include <HardwareSerial.h>
SoftwareSerial mySerial = SoftwareSerial(8,7);
void setup(){
mySerial.begin(57600);
Serial.begin(57600);
mySerial.setHardwareSerial(Serial);
}
3] SoftwareSerial has a function setTo(bool). It sets the _toCompp to either true of false depending on the variable you pass it.
4] in SoftwareSerial.begin, _toCompp is set to false
So what on earth is _toCompp?
5] if _toCompp is set to false, your program will behave as it usually does (i.e serial data from a device will be written to serial buffer).
6] If _toCompp is true, it will send the serial data from a device directly to the computer (virtually eliminating the need for Arduino's serial buffer)
7] If you want to modify the format of how you send the data to the computer, you can play around with SoftwareSerial.recv() by changing the following:
if (_toCompp == true)
hardware.print(d,HEX);
if (_toCompp == true)
hardware.print(" ");
See files below for HardwareSerial.h,.cpp and SoftwareSerial.h,.cpp
HardwareSerial.h and .cpp goes to arduino>hardware>arduino>cores>arduino>
SoftwareSerial.h and .cpp goes to arduino>libraries>SoftwareSerial (note that this is the new version of newSoftSerial and you will need the other 2 files that go along with this library... Google should give you access to these if necessary)
SoftwareSerial.h (3.82 KB)
SoftwareSerial.cpp (13.8 KB)
HardwareSerial.cpp (5.91 KB)
HardwareSerial.h (2.37 KB)
dropsofjupiter89:
I know. But I am sending the images over to the computer via USB and doing some image processing there. Is this still too slow? I only need to take one pic every 3 seconds.
I want to do the same thing using an Arduino Uno and my android phone. Someone help me
u]Reading a JPEG, and send the results to the Serial port, or TFT, is possible. [/u]
What you need: an Arduino MEGA256( I did not try MEGA128 but it would probably do the job), Arduino Due, ESP8266, or other platform that has a consistant amount of RAM.
Ultimately, a SPI TFT screen will allow you to enjoy your work on the spot, but the library can work with out it.
If you have what it needs, go there:
With a ESP8266 you can transform a 320x240 JPEG in about 460ms, when clocked at 160 MHz. You could, in theory, show 2 pictures per second, if you serial camera can supply them, and still have 80ms to make your arduino go BEEP if you feel like it.
Hope this library will help. Cheer. 
Frédéric_Plante:
Are you not sicked of people telling you "this is so impossible" bla bla bla...
A little bit, but what I am really sick of, is people ressurecting five year old threads.
(All Arduinos have "consistent amounts of RAM" - it'd be really tough programming them otherwise)
I never said impossible, in fact on record as having done it (in extremely limited cases, like fixed Huffman tables), but please remember some of the platforms you mentioned were rare or non-existent five years ago
Hi there,
For those looking for challenges for the New Year, let's know it is possible to make some streaming video, in graysacle though, with an Arduino DUE, see this video :
and the complete tutorial, programming from scratch, not even the Arduino IDE :
Happy New year !!!
ard_newbie:
[...] let's know it is possible to make some streaming video, in graysacle though, with an Arduino DUE[...][...]Happy New year !!!
Happy New year to you mate! 
Can't touch this, from MC Hammer
The fact is that it can be done on a MEGA328. How ever, for network streaming, you will need a network device.
Thank you @ard_newbie for the challenge. 
