Saturday, November 16, 2013

RS4 - Robot line following feature

The robot has a new feature, it can follow a black line painted on the floor.
I've created a new sign with a line, when the robot reads this sign it will begin the line following process.

How it works

In fact the line following feature is implemented in a very simple way. Because the line is black is easy to isolate from the ground and this is performed using a simple binarization. Here are the steps of the implemented feature.

1. ROI

First thing to do is to chose a ROI (region of interest) like showed in the next picture.

In this case the middle region of the image will look like this:

Changing the ROI image (up an down) will change the robot behavior in the corners. If a top region is chosen the robot will turn sooner, otherwise it will turn later. This requires some tuning, it will depend on the robot speed and camera tilt angle.

Code looks like this:

Rect roi(0, 190, 640, 100);

2. Threshold

Next thing to do is to threshold ROI image, the threshold level has to be tuned, the ideia is to get something like this:

I'm using morphological operations to reduce noise. Code looks like this:

threshold(roiImg, roiImg, thVal , 255, 0);
bitwise_not(roiImg, roiImg); // negative image
Mat erodeElmt = getStructuringElement(MORPH_RECT, Size(3, 3));
Mat dilateElmt = getStructuringElement(MORPH_RECT, Size(5, 5));
erode(roiImg, roiImg, erodeElmt);
dilate(roiImg, roiImg, dilateElmt);

3. Find contours and center

Next step is to find image contours, in this case it will have just one contour (white quadrilateral). After finding image contour is easy to find its center that will be used to turn the robot. If the contour center moves to one side, the robot must turn to follow it.

Code to find center:

findContours(roiImg, contours, hierarchy, CV_RETR_TREE,CV_CHAIN_APPROX_SIMPLE, Point(0,0));
for (size_t i = 0; i < contours.size(); i++) {
float area = contourArea(contours[i]);
if (area > 2000) {
Moments mu;
mu = moments(contours[i], false);
Point2f center(mu.m10 / mu.m00, 240); // point in center (x only)
circle(camera, center, 5, Scalar(0, 255, 0), -1, 8, 0);


  1. Hi,
    Few questions I couldn't find in the posts:
    1. What resolution are you using for OpenCV image recognition?
    2. What are average frame rates for both experiments: line following and markers?
    3. Do you use RasPi A or B? You stated that B but also stated that you have 256MB of RAM which is for Model B.


  2. Hello Michal,

    1. I'm using 320 x 240 resolution.

    2. No sure of the correct number, line following I get around 15 fps, marker reading around 10 fps. I have to get a correct measure for each task and I'll update the blog with his information.

    3. It is a model B 256 MB RAM. There are two model B, 256 and 512. It is working at 800 Mhz (first overclocking step).

  3. Thanks for the update. I had in mind similar robot for quite some time, so I'm looking forward to next posts.

    BTW, any chances for github repo?

    1. I'll think about it. At the moment I don't have time, I'm starting a new job.

  4. Hi Samuel,

    Its your line follower code open source ? maybe we can add some improvements, im working on a similar project.

    Best regards,

  5. Hello, i'm french student and I work to built a line following robot . I use a Rasperry Pi with Pi camera and Adafruit 16 channel. But my method i choice is to create 4 point on the picture and take the color ( black or white / 0 1 ), but this method is not precise and i want to use your method. It possible to send code source of the correction of line following with OpenCV the Factor with PWM please ? . Sorry for my bad english ..
    You can send the source code on, or part of source code.

  6. Thank you Samuel, your tutorial is pretty good

  7. First of all great work. I'm also trying to build one like this one. I have managed to detect isolate contour and center of the feature. I am stuck at translating the center and map it to robots movement in x, y coordinate. Will it possible for you to share the logic. I have seen something about servo angle in your another post. Will you be kind enough to elaborate?

  8. samuel hello, congratulations for the project.
    I want to start building a balanced card but do not know where to start.
    you can tell me where I can find ideas for softwer on PIC24FJ64GA002 and related patterns.
    I hope in your help thanks
    Forte de Massimilinao

  9. hi...I am a newbie to Raspberry Pi and want to make a line follower using image processing....can u send me the code and libraries....Thanks

  10. Hi Samuel,

    Thank you for this post. I employed your method however the pi is identifying the regions to the left and right of the line as separate objects! So the largest contour is not the line itself but the area either to the right or the left of it. Do you have any suggestions? Is this how the findcontours function works? i.e. if you have a single object in the image, it also the detects the rest of the image as an object/ multiple objects?

    Thank You,

  11. Hi,
    I would like to study and adjust your code for my own project
    would it be possible for you to give me a copy of your code

  12. Samuel boa noite. Poste muito bom.
    Poderia me enviar todo o código do robô para o meu e-mail?

  13. This comment has been removed by a blog administrator.

  14. Hi!

    Very nice job! I am interested in line tracking programming on the Raspberry pi. I would like to try this code in my project. Please, It's possible. Could you give this code me?


  15. hy sir its amazing , can you buy this project????

  16. Can u mail the code at plz its a request i am working on a similar project and i have some problem with the code