Project 6: Following the Right Track

Robots are getting more and more powerful nowadays. They can do almost anything high on top of the mountain and deep into the sea. Look at them! Some of them serve well as waiters in restaurants. Some work hard as "couriers" in factory workshops. Some work as safety inspectors in power grid hubs with high sense of responsibility....

If you look closely, you will find that there are lines on the ground of where these robots work. And they are following the lines. Line tracking? How do they do it?

HuskyLens also has the function of line tracking. Imagine, if some guide lines are pasted at home, can Maqueen Plus become our life assistant like the food delivery robot in the robot restaurant?

Let's explore by DIY a Maqueen Plus that follows the "right track"!

Function Description:

This project uses HuskyLens line tracking function to make Maqueen Plus follow the line track on the ground.

Materials Checklist:

Knowledge Extension:

If we want Maqueen Plus to move according to the lines on the ground, we need some sensors to identify these lines. According to the different sensors, there are several kinds of line tracking methods. This project uses the line tracking function of the visual sensor ——HuskyLens to implement the line tracking effect.

I. What is Line Tracking?

Line tracking refers to the process of object moving along a designated route. A fully functional line tracking robot uses a mobile robot as a carrier, a visible light camera, an infrared thermal imager and other detection instruments as a load system, a multi-field information fusion of machine vision, electromagnetic field, GPS and GIS as a navigation system for autonomous movement and tracking of the robot, and an embedded computer as a software and hardware development platform for the control system.

II. The Comparison of Two Commonly Used Line Tracking Methods

III. The Principle of Line Tracking Function

HuskyLens line tracking function is based on Pixy, an open source project of Carnegie Mellon University.

Pixy's algorithm can recognize the color of pictures. Its basic idea is to use the color space to remove the background that all users are not interested in and extract the foreground (such as lines).

Lines/Background, extract lines and color bars, remove background color

How can Maqueen Plus follow the black line on the tracking map (which has black lines on white ground)? In fact, we only need to know the relative position of Maqueen Plus to the black line. It can be divided into the following three situations:

1. When Maqueen Plus is on the right side of the black line, control Maqueen Plus to turn left;

2. When Maqueen Plus is relatively center aligned with the black line, control Maqueen Plus to go straight;

3. When Maqueen Plus is on the left side of the black line, control Maqueen Plus to turn right.

How should this be implemented? We stripped out the information displayed on the screen by HuskyLens during the line tracking and abstracted it into the geometric mathematical model shown in the picture below.

The resolution of HuskyLens screen is 320×240. The O point in the upper left corner of the screen is the origin of the screen's coordinates (0, 0), the horizontal right direction is the positive direction of the X axis, and the vertical down direction is the positive direction of the Y axis, so the coordinates in the lower right corner of the screen are (320, 240).The dotted orange line in the above picture is the central axis of the screen, and the abscissa value of this line is 160.The black line in the screen above is the map line "seen" by HuskyLens camera. The blue arrow is the line direction calculated by HuskyLens. The starting point coordinates of the blue arrow are (x1, y1) and the ending point coordinates are (x2, y2).

To sum up, we only need to judge the position of the starting point of the blue arrow relative to the central axis to implement line tracking.

IV. Demonstration of HuskyLens Line Tracking Function

1. Object Learning

This function can track lines of specified colors and make path prediction. The default setting is to track lines of only one color. This project will be explained by taking line tracking of one color as an example.

Operation and Setting:

① Dial the function button to the left or right until the word "Line Tracking" is displayed at the top of the screen.

② Long press the function button to enter the parameter setting of the line tracking function.

③ Dial the function button right or left until "Learn Multiple" is selected, then short press the function button, and dial it to the left to turn off the "Learn Multiple" switch, that is, the square icon on the progress bar is turned to the left. Then short press the function button to complete this parameter.

④ You can also turn on the LED by setting "LED Switch". This is very useful in the dark environment. Referring to the above method, turn on the "LED Switch".

⑤ Dial the function button to the left until "Save & Return" is selected, and short press the function button to save the parameters and it will return automatically.

2. Learning and Tracking

Line Learning: Point the "+" symbol at the line, then point the orange box at the background area. It is recommended that within the view field of HuskyLens, just remain lines to learn and no any cross lines. Try to keep HuskyLens parallel to the target line, then HuskyLens will automatically detect the line and a white arrow will appear. Then short press the "learning button" and the white arrow turns into a blue arrow.

Line Tracking: When HuskyLens detects the line which has been learned (i.e. lines of the same color), a blue arrow will appear automatically on the screen. The direction of the arrow indicates the predicted direction of the line.

* 1. When learning the line, we need to adjust the position of HuskyLens to be parallel to the line.

* 2. HuskyLens can learn line in any color that have an obvious color contrast to background, but these lines must be monochrome lines with obvious color contrast from background to keep line tracking stable.

* 3. HuskyLens can learn and track multiple lines according to the different colors of the lines, but the lines must be monochromatic lines with obvious color contrast from the background. In most cases, only use line in one color during line tracking. Therefore, in order to ensure stability, we recommend to track the line in single color.

* 4. The color of the lines has a lot to do with the ambient light. When patrolling the line, please try to keep the ambient light as stable as possible.

Project Practice:

We will implement the project according to the line tracking logic and continuously optimize the line tracking effect in several steps so that Maqueen Plus can pass the map quickly and stably. First, we will learn to use HuskyLens line tracking function, read the abscissa data of lines, write a simple program to adjust the motion state to meet the line tracking requirements, and then improve our project program according to the debugging result.

Task 1: Line Tracking Algorithm 1

1.Structure Construction

Using screws to fix HuskyLens and Maqueen Plus, it should be noted that we need to adjust the camera obliquely downward for line tracking, so that Maqueen Plus can see the black line closer and the tracking effect would be better.

2.Program Design

When HuskyLens detects that the black line is on the left side of the screen, i.e. the abscissa value x1160, control Maqueen Plus to turn right; When the black line is in the middle of the screen, x1=160, control Maqueen Plus to go straight.

3.Program Example

4. Execution Result

Under the HuskyLens line tracking function, learn the line and background, and line tracking function will work as long as the camera downward.

Maqueen Plus can complete the basic line tracking tasks, but at the same time it also exposes the following problems:

(1) Maqueen Plus obviously sways left and right while moving forward, the speed changes are not consistent, and the straight movement isn’t stable;

(2) The speed cannot be set too fast or it is easy to offtrack at the corner;

(3) Different bends require different turning speeds. When there are several bends in the tracking map, it is easy to offtrack.

* If Maqueen Plus depart from the line during the turn, it needs to adjust the speed of the left and right motors continuously.

How to optimize it? Let's have a look!

Task 2: Line Tracking Algorithm 2

1.Program Design

In Task 1, Maqueen Plus sways left and right in its forward movement and cannot steadily move straight. Why is this happened? Because the horizontal axis domain of Maqueen Plus straight moving is only a line, and there is inertia in the movement process. So keep straight moving at x=160 is difficult to achieve.

The optimization idea is to expand the straight movement domain. As shown in the following picture, we set the domain [150,170] as the Maqueen Plus straight movement domain. When the starting point coordinate x1 is within this domain, control the Maqueen Plus to go straight. When x1170, control Maqueen Plus to turn right.

2.Program Example

On the basis of program of task 1, the main program is unchanged, and the function "line tracking" is modified as follows.

3.Execution Result

Through these adjustments, we find that the line tracking effect is better, and when it encounters an arc, it can almost stay on line. However, it is still easy to offtrack in tracking maps with large changes of bend.

Task 3: Line Tracking Algorithm 3

Try to prevent the car from offtrack when entering the bend.

1.Program Design

Let's continue to adjust. Since we can divide the detected black line into three domains, can we continue to divide the domains into five as we did?

The closer the black line is to extreme sides, the faster the turning speed is. The closer the black line is to the middle, the more it tends to move straight.

Or further, can it be directly divided into 7 speed regulation domains?

2.Program Example

On the basis of the task 2 program, the main program remains unchanged and the function "line tracking" is modified as follows.

3.Execution Result

The line tracking speed of Maqueen Plus now can be faster, and the speed change is much smoother no matter turning or moving straight.

Project Summary:

Project Review

In this project, we implement the line tracking algorithm from simple to complex, making the effect of line tracking faster and more stable.

Knowledge Nodes Recap

1.Learn the main ideas of line tracking

2.Learn the operation method of HuskyLens line tracking function

3.Optimization Method of Line Tracking Algorithm

Project Extension:

After completing the line tracking without crossing, it may be necessary to handle the crossing line tracking as shown in the following picture. Can the tag recognition be used here to identify and let Maqueen Plus choose the correct path at the intersection?

Knowledge Expansion:

The motion status or line tracking domains of Maqueen Plus can be divided into 2 gears, 3 gears, 5 gears and 7 gears during line tracking. Is it still possible to further subdivide? 9 gears?11 gears?......

until the infinite subdivision. The more subdivided the speed regulation domain is, the better the line tracking effect is. But programs will get longer and longer. Any solution?

The PID speed regulation algorithm can help us solve this problem. PID stands for portion, integral and differential control. It is a closed-loop control system. Closed-loop control is a control method that corrects according to the feedback from the controlled object. It can correct according to a certain standard according to the Error between the measured actual value and the planned value.

If you are interested in it, you can search for relevant information on the internet and continue to optimize our line tracking effect.