Course 5 / Lecture 5:

Robot control using voice commands
(speech recognition)

In this lecture you will learn how to use voice commands to control a robot. You will learn how to setup a text to speech connection, that can convert voice into text. The next step is to use the text messages received in the robot control code, to do various tasks. This is one of the most enjoyable lectures of this website.

Figure 1
Figure 1 - Control your Sniffer using voice commands


Instruction video

Lecture video - How you can control your robot using voice commands

Task #1:

Install a Speech to Text connection in Control Panel and set up the detectable words in it!

Step 1: Install a Speech to Text connection

To control the robot with your own voice, first you need to install a Speech to Text connection that can recognize the words you say using your microphone. After that, in the Snap application you can write code to make some action with the robot depending on the recognized word. The way you can create it is the same as the other connections. In Control Panel, click on Create new Connection, then select the Audio method and choose the Speech to Text option as Figure 2 shows.

figure 2
Figure 2 - Installing Speech to Text connection

At this point of the creation, you can see two types of Speech to Text connection: Browser and Server. To use this technology in your browser, please select the Browser connection. In the connection details (Figure 3) you can name this connection, set the accuracy level of the speech recognition and also you can turn on or off the notifications which would inform you in case of a recognized word. After you have set up the connection, please click on the 'Ok' button.

Figure 3 - Connection details of Speech to Text connection

Step 2: Create a new detectable word set

In the newly created connection, you need to set up all the words that you want to use for controlling the Sniffer. In the Speech to Text connection, select the 'Detectable words' tab and click on Create new Detectable word button. Here you can add simple words or select from the predefined set of colours, directions or numbers. The Directions set (Figure 4) contains all directions that you need to control the robot. After you have added all the words that you need, you have to turn on the connection because this is the way you can initialize the detectable words that you added.

figure 3
Figure 4 - Create a list of detectable words

Task #2:

Create a Text to Speech connection in Control Panel!

The second connection that you are going to create in this lecture will provide you the functionality of getting a sound confirmation of the completed task by the robot. To make it happen, please follow the same route as it was in the previous task by clicking on Create new Connection in Control Panel, and after selecting the Audio method now you should select Text to Speech option as you can see on Figure 5. This connection used for converting incoming texts to voice messages.

figure 4
Figure 5 - Installing Text to Speech connection

After you selected the Text to Speech option, please choose the Browser type of connection to use this technology in your web browser. Before finishing the installation of this connection, you can set a connection name for it and select the language of the speech conversions. Finally, you can also select the speed of the voice so it can speak as fast as you want (Figure 6).

Figure 6 - Set connection details of Text to Speech

Task #3:

Write a program that subscribes to the Speech to Text connection and creates a variable to store the voice message!

Step 1: Subscribe to the Speech to Text connection

After you set up all the connections, you can write the code in Snap. The first thing that you need to do is to drag the 'when green flag clicked' block. Next you need to subscribe to the Speech to Text connection by dragging a 'Subscribe' block from Ozeki. Using this block, the Snap can receive all the messages sent by the connection. In the 'Subscribe' block, you can select that connection from the drop-down menu, so please choose the Speech to Text connection you installed before like on Figure 7. Now, the Snap application can receive all the spoken word as an incoming text messages.

figure 5
Figure 7 - Subscribe to the Speech to Text connection through Snap

Step 2: Create a variable to store the messages coming from Speech to Text

The next block that you need to add is the 'Get next message' block. The role of this block is to wait for incoming messages from the subscribed connection which this time will be the Speech to Text connection. Now you have to create a variable where you will be able to store a message one at a time. Choose the Variables category and click on the Make a variable button. In the popup window, name it as 'direction' and check the 'for all sprites' option and press 'Ok' like Figure 8 shows.

Figure 9
Figure 8 - Making a variable called 'direction'

Step 3: Store the incoming message in the variable

Drag the 'Set [variable] to message value (string)' block from the Ozeki category and select the 'direction' variable from the drop-down list for the 'variable' field of the block to save the connection's messages into the 'direction' variable like Figure 9 demonstrates. The 'Get next message' block waiting for the next incoming message, and if a message is received, it is stored in the 'direction' variable due to the previously placed 'Set direction to message value (string)' block. At last, please add the 'Unsubscribe' block from the Ozeki category to stop listening to Speech to Text connection.

figure 10
Figure 9 - Saving the incoming message into the variable

Task #4:

Upgrade the program to compare the messages with directions!

You can compare the stored message in the 'direction' variable to predefined directions. For that, you have to add 'if [condition] [blocks]' blocks and drag '[variable] matches [text]' ones from the Ozeki category into their condition fields where you should select a variable and enter a text that you want to compare with each other. So please select the 'direction' variable for the 'variable' field of the block and in the 'text' field just type a direction (e.g. forward and backward) that you want the variable to be compared with. This 'matches' block is a predicate type of block that returns 'true' value if the value stored in the selected variable matches with the entered text, otherwise false. The blocks in the 'if' block will be executed if the matching 'match' blocks become true.

figure 7
Figure 10 - Comparing the messages

Task #5:

Upgrade the code to move the robot to the requested direction!

If your voice message matched with a direction, you need to specify the task that the robot should do. In this lecture, the robot will do a really simple tasks. In case of matching, the robot will move to the given direction. You can do this using a 'Drive [motors] [direction] [time] sec at [power] %' block found in Lego motors category where you have got plenty of options to choose. The first parameter is the driven motor of the robot. The second parameter is the movement direction. In the third parameter, you can also assign the movement time in seconds. The last parameter is the power percentage of the motors. Set them like on Figure 11 and start the program by clicking on the green flag. This will move the robot forward or backward depending on your voice command.

figure 8
Figure 11 - Moving the robot using voice commands

Task #6:

Upgrade the code to report using Text to Speech connection if the task is completed!

When the robot completed its movement, it needs to inform you about the completion. That is the reason why you created the text to speech connection. For using it, just add a 'To [connection], message: [text]' block from the Ozeki category where you should select the 'My_text_to_Speech_1' connection from the drop-down list for the block's connection field and type a message into the text field to read it out loud that you will hear in case of a successfully executed task (Figure 12). The message will be passed to the connection and you will hear the result from your computer's speaker.

figure 9
Figure 12 - After the motors moved, it says 'Task completed!'

Task #7:

Upgrade the program to read multiple voice commands!

In the previous tasks, you created the Snap program that is capable of receiving a voice message, match with control commands and send a message to you in case of completed task. But it can only do that one time, and you have to start again the Snap program to wait for the next voice command. To avoid this, just attach a 'forever' block after the 'when green flag clicked' and fill it with the other blocks you placed before as you can see on Figure 13. With that your program can handle multiple voice commands in one running. It runs until you stop it by clicking on the red stop button.

Figure 13 - The program code upgraded to read multiple voice commands

Program code

Figure 14 - The complete program code

More information


Copyright © 2000- |Ozeki Ltd | |
Page: 6048 | | Login
Thank you for visiting this page