Pick and place robot with barcode scanner and opencv

[josheng17]

Hi all,
This is only my second 'large' Arduino project and firstly I don't expect to do it all my self, so will be hiring people for the two key areas that are beyond my ability
(at least at this stage), the bar code database and the opencv work.
But for starters let me briefly explain the project...
This is a production line/ pick and place project, with many 'stations' each doing individual tasks.
I have already completed the hardware for almost all of the stations (over 1000hrs of 3D printing later...) and have a widish variety of suitable electronic components to use, although for my areas of concern I'm happy to purchase whats required.
I'm confident with basic Arduino coding, wiring, etc. and the initialization/homing procedures I have already finished successfully myself so will not require any wiring/schematic help, etc.

The two parts I need to outsource again are the bar code database and opencv code. But before i do this I'd love to get a better understanding of how to do this and whats required.

Firstly, the opencv part i'm fairly confident I should be able to integrate with few major difficulties. It's only used at one 'station' and would do a similar (albeit more complicated/specific in terms of vision identification) job to this:

Pick and place Robot using image processing OpenCV - YouTube (not mine)

There would be Arduino processes running before and after the opencv part which would act to effectively interrupt the Arduino processes by sending it 3 sets of information, X&Y coordinates (to control stepper motors) and an angle (to control a servo) when id identification criteria are met. I think this can be done but I'd love to know any potential issues or problems i might encounter when hiring/coding for this.

The bar code database part however has me stumped. I've googled this for months while continuing work on the other parts, trying to find a solution but haven't come across one yet.

Each station requires this. Ultimately this is what needs to be accomplished:

As the product passes by each station it's bar code is scanned I've got this module for starters: https://www.aliexpress.com/item/1005001458273140.html?spm=a2g0s.9042311.0.0.27424c4dbUfGFz)

I then need it to use that bar code number, lets say '123456' to recall information from a table, relevant to that station. For example, the simplest station merely sprays the product as it goes by, I have a relay that activated the pump that can be turned on and off.

So the arduino would read the bar code then go to the 'spray station' column (Y) for that barcode number (X) and read a value, lets say '20000.' If 20000ms has expired since the previous 'spray' then the product passes without further interaction. If the time elapsed is greater than 20000ms then the relay is switched on briefly to spray the product as it passes.

Another station might be the blowing station, similarly simple, a relay turns a fan on and off as it passes. Again, the arduino would read the bar code then go to the 'blower' column (Y) for that barcode number (X) and read a value, lets say '10000.' If 10000ms has expired since the previous 'blow' then the product passes without further interaction. If the time elapsed is greater than 10000ms then the relay is switched on briefly to blow air at the product as it passes.

So, many bar codes and may different columns for various stations, each doing unique processes to the product as it passes.

Ideally i would like to be able to add and edit table entries online while never having to turn off/update the various stations. But if that's not possible, the nearest possible solution would suffice.

Also, data logging to an online platform is required, ie. logging the times/dates of each action that is successfully completed as well as a few sensor input values.

I hope that's clear enough for a start, I'll post a couple of photos of one particular station (the one that requires the computer vision (for which I currently have a jevois to hopefully give you a better idea of the project)

[josheng17]

My pick and place 'station' and its control box. (its a bet messy inside as i'm waiting for a CNC shield to clean it up, but functions perfectly!)

[josheng17]

[lastchancename]

I understand reading the barcodes when you arrive at an item location, but shy read every barcode on the way?

Wouldn’t it be easier to ‘know’ where you’re going, then verify the item once you get there, and confirm or alert - if it’s wrong..?

It just seems faster to accumulate a target database, then use that to plan your journey, rather than to stop at every house and read the letterbox and street signs for navigation every few steps !

[josheng17]

Hi Lastchancename!
thanks for your reply, I've read it a few times but I"m not entirely sure what you mean except that i haven't explained it properly.
Imagine a physical loop conveyor belt with a few 'stations' along the way, ie a spray station.
Someone could place an object on it at any point (with a bar code). There would be corresponding details in the data base specific to that object, ie. spray every 24 hrs (lets say for arguments sake that the physical conveyor loop takes 1 hr to complete, so 1/24 bar code scans will result in a spray). Another object could also be placed on the physical conveyor loop (with a bar code) that requires no spraying at all, hence, its bar code is read but no actions take place at the spray station and it resumes its journey indefinitely around the conveyor loop. There wouldn't be any 'list' of objects in their respective order as they would be added and removed via an external process.
Sorry if my language is inadequate, self taught

Does that clarify it at all?

[josheng17]

here's a sketch of the physical environment (i prefer drawings!) that i hope will clarify it.
So it the object with barcode number '1234' passes the station for the first time it will be sprayed. If it passes the station again but 10,000ms HASN'T yet passed it will go past without a spray. If if passes yet again but 10,000ms HAS passed, it will be sprayed again.
Similar for the object with barcode number 1235 but every 20,000ms miniumum. The object with bar code number 1236 would never be sprayed, regardless of how many times it passes the spray station.

[lastchancename]

Ok for me, It isn’t really a picking system which goes to find a product, but rather identify a passing object, and then decide if it has been seen before, and whether it needs to be ‘serviced’.

That’s reasonable, so you need to keep a FIFO list of items in order of appearance, along with a time stamp of when they were last seen.

Using that, you can determine if it’s the first appearance if an item, or how long since it was last seen.
You might keep a flag to say if it’s already been ‘sprayed’ to avoid overdosing and flag for removal.

Then, at some point, all your locations will have been serviced, and there’ll be no more capacity to load more.
You can alert the operator, and/or search for unserviced items, clear the ‘last seen’ record, and carry on.

There may be some gains by developing a mechanism to determine when a bay has a new item loaded - to make the discovery of new items faster.

[josheng17]

Ok for me, It isn’t really a picking system which goes to find a product, but rather identify a passing object, and then decide if it has been seen before, and whether it needs to be ‘serviced’.

Spot on

Then, at some point, all your locations will have been serviced, and there’ll be no more capacity to load more.

Correct. But some objects would be removed manually (and constantly) (so this i'm happy to rely on a human input at this stage, ie. If i see the conveyor belt is fully loaded I know i can't load more objects untill others are removed.

For the rest of your reply however I will have to do some research and learn about the terms and code you're referring to and get back to you.

Thanks!