Robot control based on visual input (Build a line following robot)
The line following robot is a classic example of robot control. You task in this lecture will be to use the color sensor as input and develop the algorithm to follow a line. Of course you will be presented with a recommended strategy an information on how others have achieved this goal in the past.
Figure 1 - Lego Robot and the test track
- Ozeki Dozer: Ozeki Dozer build instructions
- Ozeki 10 installed: Installed Ozeki 10 Robot OS
- Lego connection installed: Installed Lego bluetooth connection
- SNAP basics: Basic knowlege of the Snap programming language
- Color sensor reference chart: Download the chart
- Test track
Lecture video - The robot follows the line
Describe what the Color sensor is!
Color sensor - it is a sensor which can measure the light intensity and determine which color it sees. It has 3 modes: Color, Reflected and Ambient light intensity.
Get to know the Color mode and test it using the reference chart!
The first operation mode you will use is Color mode, in which the sensor can detect the color of surfaces about 1 centimeter (0.39 inches) away. The sensor is mounted pointing straight down so it can detect the color of the surface beneath the robot. To test the color measurement, download and print the color reference chart and position the robot on top of it. Then go to Port View app on the EV3 Brick to see the detected color, displayed as a number. The sensor can distinguish among black (1), blue (2), green (3), yellow (4), red (5), white (6) and brown (7). The number 0 indicates that the sensor can not detect any color, which may mean that the surface is too far away from the sensor or too close to it. On Figure 2, you can see that the color sensor detects white color.
Figure 2 - Go to Port View app to see the sensor measurement
Create a program which stays the robot inside a colored line!
In SNAP, you can use Wait, Loop, and Conditional blocks to make decisions based on the sensor value. For example, you can make the robot drive around without going outside the blue outline as shown on Figure 3. To do this, the robot should drive forward until it sees the blue line. Then, it should back up, turn around, and move forward in another direction. But first, you need to create the circular test track that you will use to test the sensor. In the video and some figures below, we used blue tape to build the test track, but using a black tape is a better alternative. So ceate your own track using some black tape and a light-colored surface.
Figure 3 - Lego Robot and the test track
Figure 4 shows the program flow you will need to make the robot drive around without leaving the black shape. The program waits until it sees the black line. For this program, you need to use the Color sensor in Color mode. To check it, open the Control Panel, select the connection of the sensor and choose the Configure tab. You can see its actually set mode in the Sensor mode groupbox.
Figure 4 - The program flow of the StayInCircle program
In this program, you can see a 'forever' block that repeats the loop core. In the loop core, there is a 'wait until' block. When the Color sensor sees black, the block stops waiting, and the program moves on to the next block. Figure 5 shows the finished StayInCircle program with the 'wait' block to wait for the black line. Place the robot inside the black outlint on the test track, and run the program to see it in action.
Figure 5 - StayInCircle program code
Create a program which controls the robot to follow a line!
In your next project, you will use the Color sensor to create a line-following robot, a robot that follows the line on your test track. Let's look at the strategy behind this program. When the robot is following a black line on a white mat, the sensor will always detect one of two colors: white or black. Therefore, to create a line-following program for a black-and-white environment, you can use a 'if-else' block that looks for the color black. When the sensor sees black, the 'if-else' block triggers a 'Drive' block to preform one movement: if it sees another color (white), it preforms a different movement, as shown on Figure 6. If the robot sees white, it will not know which side of the line it is on. You need to make sure it always stays on only one side of the line; otherwise, it will stray off the line into the white area. You can do this by always driving Ozeki Dozer right when it sees black and left when it sees white. If you make the robot drive forward a little as it steers and if you repeat this behavior indefinitely, the robot will follow the line (zigzag movement).
Figure 6 - Color Sensor seeing black and white
The ColorLine program implements the strategy presented on Figure 7. Now let's have a look at the robot's behavior as it follows the line. Notice that the robot actually follows the edge of the line. The program keeps adjusting the robot's steering to the sensor measurement, causing it to zigzag across the edge: As soon as it sees the black line, it tries to move away from it by going right; as soon as it sees the white area, it tries to move back to the line by going left. As result, the robot keeps the line to the left of the sensor. This means that if you place the robot on the line such that it follows the circle clockwise, it traces the inner edge of the circle; if you make it drive around counterclockwise, it follows the outer edge. To see this, turn the robot around manually while the program runs to make it follow the line the other way; it should start following the other edge of the line. If your robot strays of the line in corners or on sharp curves, make it drive slower (choose between 15% and 25% of motor speed, it will be enough).
Figure 7 - The Dozer following the line
The ColorLine program starts with a 'when green flag clicked' block. Then there is a block to set the motor speed to 20%. The next block is a 'forever' block. You should put a 'Rotate [A+D] by °' block to the 'forever' one to move the robot slowly forward. Now comes an 'if-else' block. Put an 'X = Y' block into its condition field where X should be a 'Color [sensor]' block and Y should be a text 'black'. Insert a 'Rotate [A] by [-10] and [D] by °' into the 'then' branch and a 'Rotate [A] by  and [D] by [-10]°' into the 'else' branch. So the program moves the robot forward, then checks whether the color sensor senses black or not. If the sensed color is black, it will turn the robot to the left a bit. If not, it will turn the robot right a bit. So the robot will follow the edge of the line in a zigzag movement.
Figure 8 - The ColorLine program
Create a program which sets the LED's color based on the Color sensor!
The 'if-else' block in the ColorLine program makes the robot go right if the sensor sees the black line. If it sees any other color, such as green or red, it goes left. The ShowColor program in Figure 9 changes the brick status light color based on the Color sensor measurement. There is four cases in three embedded 'if-else' blocks, each containing a 'Set LED' block. Each case corresponds to a different color measurement: The status light turns green if the sensor measures green, red if it measures red, and orange if it measures yellow. If no color is detected, the status light turns off. But what happens if the sensor sees black, blue, white, or brown? If none of the case match the sensor value, the program executes the block inside the innermost 'if-else' blocks's 'else' brach, which is a 'Set LED [off]' block. Create the program now and test it with color reference chart.
Figure 9 - The ShowColor program
Get to know the Reflected light intensity mode and create a program which controls the robot to follow a line more smoothly!
The Color sensor can also measure the brightness of a color in Reflected Light Intensity mode. For example, the sensor can see the difference among white, gray, and black paper by shining a light on the paper and measuring how much light is reflected. The Reflected Light Intensity is measured as percentage from 0% (very low reflectivity: dark) to 100% (very high reflectivity: light). Black paper does not reflect much light, resulting in a measurement below 10%. White paper can result in a measurement higher than 60%. To verify these values, go to Port View on the Brick, choose the port the color sensor connected, press the middle button of the Brick and select COL-REFLECT like Figure 10 shows. Place the robot on the color reference chart that you printed, and observe how the sensor value changes as you move the robot over the bar with various tones of grey. Super Reflector: You should be able to find at least one material that result in Reflected Light Intensity value as high as 100%.
Figure 10 - Reflected light intensity
The advantage of the Reflected Light Intensity mode is that the robot can measure not only black and white but also a mix of black and white as it moves across the edge of the line. If the robot sees white in the ReflectedLine program, it makes a sharp turn to the left to get back to the line. But sharp turns are not necessary if the sensor is close to the line. If the robot measures light grey, a soft left turn might be sufficient to return to the line. This makes the robot follow the line more smoothly, rather than bouncing left and right repeatedly. Soft turns are not always enough to return to the line, though, so the robot still needs to take a sharp turn when it's far off the line (when it see white). To determine whether the robot measures black, dark grey, light grey or white, you need two additional threshold values, midway between the three known values, as shown on Figure 11.
Figure 11 - Threshold calculations
In Color mode, the sensor could see that the test track was either black or white. Now observe the reflected light measurement as you drag the robot with your hands (very slowly!) from the black line onto the white area of the test track. You will notice that the value gradually increases from around 6% (black) to around 62% (white). When the sensor partly sees the black line and partly sees the white paper, the value will be between these two examples, as if the sensor were seeing a grey surface. You can use this more detailed measurement to improve your line-following robot. To tell your robot what measurement you consider to be the white area or the black line, you will define white and black in your program using a threshold value. You will consider a measured value greater than this threshold as white and a measurement lower than this threshold as black. In other words, you take dark grey tones to be black and light grey tones to be white, as shown on Figure 12.
Figure 12 - Threshold calculations 2
You will now create a new line-following program that uses Reflected Light Intensity mode and a threshold value to determine whether the sensor is on the black line or on the white area. As before, the robot should turn right if it sees the black line and left if it sees white. To accomplish this, change the sensor mode to Reflected Light Intensity mode in the Control Panel. Modify the ColorLine program to make decisions based on reflected light intensity instead of detected colors. You want the robot to turn to the right if the sensor sees black, which is when the sensor value is less than the threshold, so you can choose that as the condition that triggers the upper set of blocks. Create and run the ReflectedLine program now (see Figure 13) and test that it preforms the same way as the ColorLine program.
Figure 13 - The ReflectedLine program
Figure 14 - Program codes that can sense colors and follow lines
Download the programs|
These programs are capable to make decisions based on detected colors and reflected light intensity.
It is easier to do the steps if you have the program in front of you:
Download the SNAP program codes
- Drive your robot along a square
- Learn to use keyboard events
- Move your Lego robot from the keyboard
- Learn to use the gamepad
- Use a gamepad to control your Lego robot
- Use sensor input in robot control
- Learn to use states in robot control (Build the safe)
- Robot state control and feedback, using the IR remote controller
- Synchronized motion