This is my first post on the forum so apologies if this is in the wrong category or whatnot. I'm currently working on a research project that involves counting the number of a specific type of beetle (Corn Rootworm if you're interested) that are on traps set up in corn fields. The professor I'm working under is insistent that I figure out a way to do this on the Arduino camera device that he already has set up. I've developed code in python that can do a pretty decent job of counting the beetles but it is not incredibly efficient and I have no way to get it to run on an Arduino. Everything I've seen in my research is people saying to use RPi instead of Arduino for anything image processing related but as my professor refuses to go down that route and I'm kinda stuck. Luckily the traps are all a distinct yellow color and the devices should be able to be set up pretty consistently so at least the background will be fairly similar but right now I'm just trying a color-based approach because a neural network seems even less feasible on Arduino. Does anyone have experience trying to get image processing to work on Arduino and if not is there something you can point me towards to convince my professor this is not the way to go about the project? Below is a picture of generally what the traps will look like and the camera used produces 640x480 JPEG images (This is just a picture stolen off of google because I don't have actual sample images from the camera).
I should note that because of the location in fields in rural areas, the Arduinos need to be able to do this independently and just report the number of bugs back to the base station via radio. I haven't looked into it much but he claims that there is no way to send the images back to the base station fast enough to have it do the processing while the Arduinos just take pictures.
Consider the ESP32-CAM module, which can be programmed by the Arduino IDE. It is inexpensive (read: dirt cheap) and offers built in WiFi and Bluetooth connectivity. The camera is decent and there is more than enough memory for some form of imaging processing.
I've seen some mention of that board, is it powerful enough to do the image processing on-board or is it supposed to take pictures and send them to a more powerful computer for processing?
I could just be confused about how its working but that project and others (below) appear to be all transmitting the image to a server that is processing the image using python. Given the use case with these being in a field, we won't have access to wifi or Bluetooth connections and so having a separate server actually doing the processing won't work.
ESP-WHO is an image processing development platform based on Espressif chips. It contains development examples that may be applied in practical applications.
Right now his camera device is running with an Adafruit Feather M0 Adalogger. He might be able to be convinced to move to a different Arduino device but this is what he's been wanting me to code something to run on. Right now I've been trying to hard code something rather than use a neural network because I figured a neural net would be too intensive to run on it. If it takes 30 minutes to process an image that's actually fine because they really only need to report once or twice a day to keep count of the beetles.
Thanks for the help, if I could get one working that'd be great I'm just a bit doubtful about whether I'll be able to run an accurate enough model to differentiate between other types of bugs or debris that lands on the sheets with this. The article I linked was working with 96x96 greyscale images and was still only 77% accurate on 4 types of fruits or something. I guess maybe I could set it up to break the image into pieces and analyze each one but I'm already pushing my coding experience so we'll see if I can manage it.
So it's not actually an Arduino, but it's still pretty low on RAM, likely not enough to actually load the whole JPEG at one time. @jremington's link is to a system that needs 4MB, so that poor Feather is woefully low.
I expect that you could do row by row processing and look for black pixels, and group them into regions of black and call those flies. I would think that it would be nigh on impossible to tell what kind of flies they are, or even if they're flies at all.
You've been handed a junior hacksaw and asked to fell a redwood. Good luck.
You really should take a very close look at the ESP-WHO library and examples. You probably don't need to do much if any coding to make an informed decision, but you DO need some hands on experience. And the cost is just ~$10 for the ESP32-CAM. Buy 2 or 3 in case you make a wiring mistake and fry one.
ESP-WHO is intended to work with 320x240 images, but if the camera/image quality is not high enough for your project, then obviously you need to look at much higher priced options.
From many years back-in-time, I argued with a Prof when I was working for the School of Electricial Engineering, Research and support. The PhD wanted to put a Cromemco S100 on a harvester for lettuce. A color NTSC video camera was to provide a field-of-view and a catapult arm would reach out, snip, and flip the head of lettuce if the computer determined the color was right for picking. After lots of software Hacking and mechanical tuning, the prototype was reasonably successful in bright midday sun: cutting and flipping the head of lettuce into a funnel snapped catcher and through a few mechanical areas to sort and box.
Early mornings, late afternoons, and cloudy days played havoc with the limited CPU power to process the color signals. The articulated arm would miss lettuces, make bad decisions and pick too soon. The obvious "fix" was high-lumen lamps similar to stadium lamps but once constructed and installed on the tractor, an alternator could not be sourced to power the lightning; thus a plan was hatched to pull a military style AC generator (test unit on loan from the Army Reserves) behind the tractor.
The contraption looked like something from a bad SciFi movie. I left research after a year, I have no idea of what happened, but since it was Federal grant paying the project, I suspect little came of it except maybe a research paper and patent filings.
Point is, there is no point really. Until you are free to build something during a prototype stage, one cannot think of all the bad things that could ruin your day. (I heard second hand that the lettuce harvester lowered the rabbit population more than once.)
Your PhD is playing difficult because s/he will continue to insist until you prove them wrong.
Start by collecting all current documentation (specs, code ...)
Do a power profile (mA hours, battery capacity, etc.)
Analyze the current software
-- SRAM usage max
-- Flash memory usage
From the above, develop your software usages. Could the Feather M0 support such a large creep in the project's scope?
Before suggesting alternate camera, prove the pixel resolution and field-of-vision can/cannot provide a minimum resolution for infestation species determination.
Work smart. Gather examples of potential, proven AI examples. You are being tested and you need to intelligently respond to the task; a simple "it will not work..." will not be adequate.
On the other hand, Arduino is C/C++, and it does not take much trouble for an experienced user to implement medium sized open source C/C++ applications using the Arduino IDE on ESP32.
Take the pictures with an LED strobe at NIGHT to eliminate external lighting effects. Using a ring light you should be able to eliminate shadows that would cause double counting. Once that is done, the rest is fairly straight forward. Flatten to B&W, boost the contrast and threshold filter. Resulting image should be similar:
Filter out speckle (disconnected dots) and edge detect (contour) to isolate bugs. Count "islands" with a certain median size. Send resulting count.
You need at least 2x the size of image for working RAM. Sequential processing won't work as you can't normally move around in the camera memory. To do processing the entire image is needed and a 2nd array for the results. A 3rd array might be needed for comparison to a reference.
