Hello,
I built an Arduino-based Scara Robot which should pour out drinks. For that, I need to detect the glasses. I know it would be easier to use predefined spots where you put the glasses but I don't want that. I want the robot to detect all the glasses on a tray and then pour out drinks.
I tried to use Computer Vision with a Raspberry Pi which worked great but only if the tray wasn't wet so I scraped the idea. Now I thought about mounting a HC-SR04 Sensor at the arm and scanning the height of every mm of the tray since the bottom of the glass is higher than the tray. Do you guys have another idea of how I could detect the glasses?
Will all of the glasses on the tray be of the same height ?
Assuming that you detect the height of the glasses how will you know where they are on the tray ?
Yes, all of the glasses are the same height.
The idea was that I know where the robot currently is and that I could make a type of list which stores the X and Y Coordinates of end effector and the measured height at the current position. This task would probably do a Raspberry Pi. So I have one Arduino which moves the arm and returns the position, one Arduino which measures the height and both Arduinos return their data to the Raspberry.
Are you thinking of measuring the height from above or from the side ?
What do you mean by "the X and Y Coordinates of end effector" ?
I measure the height from the top. I want to attach the sensor directly to the end of the arm and then follow a predetermined path to get the height of each mm of the tray (the tray has a fixed position). The Z height of the sensor would of course always remain the same. I would then expect 3 different values here are a few examples:
Value 1: 70mm (the distance between the sensor and the tray)
Value 2: 20mm (The distance between the sensor and the edge of the glass)
Value 3: 60mm (The distance between the sensor and the bottom of the glass, the interesting value.
When I find this value 3 I want to save it in a kind of list together with the current coordinates.
The resolution should be enough but my original question is: Is there a better solution to detect glasses on a tray from above? Here is an example image of what a camera or sensor would see:
And as you can see it's a very complicated task for computer vision because of all the reflection caused by the water (ignore everything around the tray)
I would have thought that the most interesting value would be the distance between the sensor and the edge of the glass because that will tell you the height of the glass and also potentially to derive the position of the glass
However, I suspect that you will have difficulties in using the HC-SR04 sensor to accurately detect the edge of the glass
I know that my idea may not be the best that's why I want to know if someone in this forum has a better solution.
The glasses will be randomly placed on the tray?
Yes, I know that that's a difficult problem and that there are easier solutions but I want to try it.
I'd start by placing a glass in a known position and then establish some way to define a glass at position X. Work out the searching for multiple glasses later.
This is already working, the inverse kinematics for the robot are working perfectly. I just need a method to get the coordinates from the glasses.
Yes, I tried that idea but unfortunately, it introduced other but similarly large problems.
For sensor I'd be looking at something like the VL53L0X or VL53L1X optical distance sensors. Those have a much narrower cone, so when scanning the top of the tray (using a gantry or so) and taking measurements say every 10 mm you'd get a height map. Maybe you miss the edges but I think you have enough information to deduce where the rough centre of the cups is, and that's what matters as that's where you want to pour, not on the edges.
You may even be able to do the same with the sensor in a fixed location above the tray and then angling it in two directions to scan the whole tray. Two small servos would be able to do the job. Mechanically simpler but harder to process the data. Or go really fancy and do something with LiDAR which is designed for just this type of jobs.
What @wvmarle said. Assume a 30x30 2D array. Record height readings from A1 thru A30, then B1 thru B30 and continuing thru DD30. Apply the glass detection logic to the array and arrive at the location of each glass.
Will the tray at least be in the same place each time?
Yes, the tray will be in the same position. The method you described is the same that I poorly tried to explain before. I will try your suggestions thanks for your advice.
Don't pass lightly over this.
Plus myriad references on the webs.
A different approach would be to have a pattern printed on the tray surface. An image is then taken from above, similar to the one in post #10. But with large aperture and focus on the rim. Then it's all about finding sharp circles in blurry surrounding.
1 Like
This topic was automatically closed 180 days after the last reply. New replies are no longer allowed.