RS4 - Robot Update (new features)

RS4 now can detect and read some signs and perform associated actions. In these videos it follows and reads various signs.

How it can read signs using OpenCV

First it must locate a sign and move to it, this is performed by following the blue color around around the signs. I'm having issues with color tracking because the AWB keeps changing the image lightning and at the moment I can't turn it off. The problem is already exposed in the raspberrypi.org forum and they are working in a solution.

When it is close to the sign it performs the next steps to read it:

1. Find image contours

Apply GaussianBlur to get a smooth image an then use Canny to detect edges. Now use this image to find countour using OpenCV method findContours. The Canny output looks like this:

2. Find approximate rectangular contours

All the signs have a black rectangle around them so the next step is to find rectangular shaped contours. This is performed using approxPolyDP method in OpenCV. At this point found polygons are filtered by number of corners (4 corners) and minimum area.

3. Isolate sign and correct perspective

Because the robot moves it will not find perfectly aligned signs. A perspective correction is necessary before try to match the sign with the previous loaded reference images. This is done with the warpPerspective method. Details how to use it here:

http://opencv-code.com/tutorials/automatic-perspective-correction-for-quadrilateral-objects/

In the next image you can see the result of the process, in "B" is showed the corrected image.

4. Binarize and match

After isolate and correct the interest area, the result image is binarized and a  comparison is performed with all the reference images to look for a match. At the moment the system has 8 reference images to compare.

Comparison is performed using a binary XOR function (bitwise_xor). In the next images is showed the comparison result for match and not match, using countNonZero method is possible to detect the match sign.

Match image

 Not match image

Result

This methodology works well and it is fast enough to use it in Raspberry Pi. I've tried known methods like SURF and SIFT but are to slow for real time application using Raspberry Pi.