SPY-DER : a Speech & Web Controlled Surveillance Spider Robot

SPY-DER is a speech and web-controlled surveillance spider robot. So basically, it is a robot, which can walk like a spider and work as a spy through its camera, that’s why I have given it the name “SPY-DER”. One can control this robot in two ways, firstly by giving it voice commands and secondly by a web-based control interface. So whenever I call it by its nickname 'Bumblebee', it starts listening to me, and then based on my voice command it will act. Here, I am using mainly two technologies, first one is hot-word or wake-word detection and the second one is speech recognition. And with the speech recognition, I am also using intent detection, so that I can give it the same command in different ways (i.e., “Wave your hands” or “say hello” both the commands will make the robot wave its legs). Coming to the web control part, one can simply open a particular URL in any browser and will be able to control the spider. The web-based interface contains all the control options and one can control the spider using the buttons availabe there. Similarly, one can open another URL and watch the live video feed from this robot’s camera. One can watch the demo of this project here: https://youtu.be/3edXTxIZ_2U

I have used the following hardware components to build this:

a. Arduino Nano

b. Raspberry Pi Zero W

c. Raspberry Pi camera

d. 5v to 3.3v Logic level shifter

e. Nano 328P Expansion Adapter Breakout Board IO Shield

f. SG90 Mini Servo 12 pieces

g. Buck Converter Lm2596

h. Lithium-Ion Battery 2 pieces

i. LEDs

j. Jumper Wires

k. USB mic

l. OTG cable

If you prefer to watch a video, then here goes the full video tutorial for this project. Also, if you have any problem in understanding any of the steps below, then also you can watch the video, where I have explained everything in details. You can find the first part of the video at https://youtu.be/fnMmnd9k6q8 .

Initially, I built a simple Bluetooth-controlled spider robot using an Arduino nano. But then I was able to control the robot only using an android/IOS app, which was not cool obviously. Thus, I decided to implement the speech recognition, web control, and surveillance part in it. Now implementing all these features using a single arduino was not possible, so obviously, I needed a small computer here. That's why I decided to build this using Raspberry Pi zero. Now the whole system can be implemented using a Raspberry Pi zero only, but in that case, I had to rewrite the whole spider movement control code in Raspberry Pi, which would be really time taking. That's why I decided to connect the Raspberry Pi with the Arduino nano through serial communication. Then the Arduino will be controlling all the movements of the spider and the Raspberry Pi will simply control the Arduino, in this way using the same code with very few modifications in Arduino, I was able to implement all these new features in this project.

All the codes for this project are written in Python for Raspberry Pi. For the web-based control part, I have used Flask framework. And the webpage was built using HTML, CSS and jQuery. For the live video streaming, I have used RPi-Cam-Web-Interface (https://elinux.org/RPi-Cam-Web-Interface), due to the fact that the latency is very low here. And for the speech recognition and hot word detection part I have used Picovoice (https://picovoice.ai/) and I have modified the code in python. I have also tried to use local speech recognition systems, but as the ram and processing power is very limited in Raspberry Pi zero, the accuracy was not that good and the latency was also very high.

While designing the robot, I have followed the the design of this existing robot https://www.thingiverse.com/thing:2901132 . I have redesigned few parts in tinkercad (https://www.tinkercad.com/) and all of those parts are available at https://www.thingiverse.com/thing:4815137 .Then I have 3d printed all those parts using Ender 3 3d printer.

After printing the parts, you need to attach SG90 servo motors with the 3d printed parts.

Then, all the 3d printed body parts need to be connected together through the servo motors.

Then, the battery and battery management system need to be attached in the body of the robot. As I have connected two lithium-ion batteries in series, I was getting around 7-8 volts. Thus I used LM2596 buck converter to reduce the voltage to 5 volts, so that using this single power source Arduino, servo motors and Raspberry Pi can be powered.

At first, Arduino nano need to be attached with the I/O shield and then all the servo motors need to attached with the I/O shield. While attaching the servo motors, make sure you have attached them according to the numbering mentioned in the servo numbering figure and circuit diagram figure.

Next, you need to upload the Arduino sketch in the Arduino nano.

At first you need to calibrate the legs of the robot. The code is availabe at https://github.com/Arijit1080/Spidy-The-Spider-Robot .The program is availabe in the Legs folder.

Next, you should check if the basic functions like forward, backward, left, right etc are working or not. For that you can use the code availabe in the folder Program1.

To know more about last two steps, you can watch this video: https://youtu.be/fnMmnd9k6q8

Then, you can upload the final code in Arduino which will work with Raspberry Pi. The code is availabe at https://github.com/Arijit1080/SPY-DER-A-Speech-and-Web-Controlled-Surveillance-Spider-Robot-using-Raspberry-Pi-and-Arduino .You need to upload the SPY-DER_Arduino.ino file in the Arduino.

Next, you need to connect Raspberry Pi zero w with Raspberry Pi camera, USB mic and logic level shifter.

If you don't know about the connection for serial comunication, then watch this video : https://youtu.be/e04br5J4UpQ

To know, how to use Raspberry Pi camera with Raspberry Pi zero, watch this video: https://youtu.be/oo0A_yRrIxQ

Then, you need to connect Arduino with the logic level shifter, so that the Arduino and Raspberry Pi will be connected serially.

You need to Install all the required libraries in Raspberry Pi and upload the codes from my Github repository (https://github.com/Arijit1080/SPY-DER-A-Speech-and-Web-Controlled-Surveillance-Spider-Robot-using-Raspberry-Pi-and-Arduino).

In this step, you also need to check if the mic and camera connected with the Raspberry Pi are working properly or not.

Also, you need to clone the Picovoice (https://picovoice.ai/) repository from Github (https://github.com/Picovoice/picovoice) and required changes to that code are available in my Github repository.

For the web-control part Flask framework is required in python and all the python & HTML files are there in my repository.

For the video streaming part, RPi-Cam-Web-Interface (https://elinux.org/RPi-Cam-Web-Interface) needed to be installed. To know more about this step, you can watch this video: https://youtu.be/yzpqEw1kEGo

Finally, close the lid after fitting everything inside the body.

To run the web control interface:

cd web_controlpython3 web_control.py

To run the speech control system:

cd picovoicepython3 demo/python/picovoice_demo_mic.py 	--keyword_path resources/porcupine/resources/keyword_files/raspberry-pi/bumblebee_raspberry-pi.ppn  	--context_path your_rhino_model

It is higly recommended that before running the codes, you should watch the build video here https://youtu.be/KkZiZggtvIU , so that you will know which codes to modify and how to run them properly.

After running these codes, you can access the web control interface from any browser using the url Raspberry_pi_ip:5010 and the live video stream using Raspberry_pi_ip:80

In this way, I have built this project. Though, one can build it in many different ways also. And also there is a space of future improvements. One can try to use a local speech recognition system in this project. Also with the camera, many image processing and AI based features can be added in this project. May be I will upgrade this project in near future, so see you there :)