Open-Source Telepresence Bot - See, Speak and Drive Via the Web

During the COVID-19 pandemic families were tragically prevented from visiting their loved ones in hospital. I thought that a telepresence robot would be the ideal solution to enable crucial contact to be restored. These robots are expensive to purchase, so I started building an open-source telepresence robot with a sub $100 price tag. This is the result.

How it works:

1. Old smartphone - provides the two-way video conferencing and web connection
2. Web socket - enables the real-time controls that drive the robot forward, left and right
3. Jitsi Meet (free) - enables the bi-directional video chat
4. LDRs - detect the driving instructions sent via the web and displayed on the smartphone screen
5. Arduino - drives the wheels, based on the driving instructions from LDRs
6. PHP, Javascript and MySQL webpage - brings it all together into an online portal and phone app

The result is a stand-alone robot that waits for a visitor to connect remotely, then the visitor can see out of the robot, be seen on the robot screen, speak and hear, as well as being able to drive the robot around.

Yes, the robot could be taller. But that's for you to do :)

1. Breadboard
2. Arduino UNO R3 (any other Arduino will work fine)
3. Old smartphone (must have a front-facing camera, run Chrome web browser and have internet via WiFi / cell)
4. DC-DC converter able to turn 12V DC into 5V DC
5. 12V mini lead acid battery (1.4Ah)
6. 2 x stepper motor NEMA-14 / 35mm (size not really important)
7. 2 x stepper motor driver based on DRV8825
8. 2 x 10uF 50v electrolytic capacitors
9. 2 x robot / car toy wheels
10. 3 x LDRs (light dependent resistors) 50 - 100k Ohm
11. 3 x 10k resistors
12. 1N4004 diode (any one capable of 12V)
13. 1 - 2m of basic copper wire to connect up the breadboard, LDRs and Arduino
14. Some kind of material for the chassis - I used Lego and Lego Technics. Could use wood or stiff cardboard.
15. A domain name and website where you can host PHP scripts with a MySQL database

1. Choose a material for your chassis

I chose to use Lego and Lego Technics because I had it lying around and its dimensions are standardised. It makes prototyping and changes easy. You could also use stiff cardboard or plywood to make the chassis. The shape is not special, it just needs to be able to support two stepper motors and wheels on the bottom, and have a castor wheel at the back.

2. Base construction

Start with the base. I drilled four holes into the Lego base and secured my stepper motors with zip ties. I then used some Lego bricks to stabilise the motors a bit more.

I created a free-rotating castor wheel from Lego and fixed this at the back. Since the two stepper motors will be providing the force, this wheel is the third leg of support, but needs to move freely. You can solve this problem any way you want.

I then build a vertical wall to stick the smartphone onto, and built a base for the 12v lead acid battery. I then built the circuit, composed of an Arduino Uno and breadboard and stuck this to a Lego base board. This was then stuck on top of the lead acid battery using Prestick / Blu Tack (a temporary sticking putty).

3. LDR sensor for driving

The Arduino drives the stepper motors by taking instructions from the LDRs, which are pressed up against the right-hand side of the smartphone screen. There are three squares which change from black to white, based on the steering buttons the driver is pushing on their end in their browser. So the LDRs need to be fixed approx 2cm apart from eachother, in such a way as to be pressed tight up against the screen so no other light can leak in from the side. I used a standard Lego Technics brick with round holes and pushed the LDRs through every second hole (see pic above). Then I placed a strip above the LDRs to hold the wires in place and block out light from the back. Put this on about 25cm of copper wire so it can be moved about freely. I fixed mine to the phone screen with an elastic band - handy because you are probably going to be turning on/off the phone many times as you test, open Chrome, turn on WiFi etc - you want the sensors to be movable.

1. Make sure your Arduino is powered off and has no battery connected to it
2. Plug your USB connector into your Arduino
3. Load up the Arduino IDE software on your computer (https://www.arduino.cc/en/software)
4. Download the file attached here "Avatar1.ino" and open this in the IDE
5. Upload it to the Arduino
6. Open up the Serial Monitor in the Arduino IDE (right hand side of top toolbar / under Tools menu)
7. You should see a log of outputs relating to the light level on the LDRs
8. Cover the middle and left LDR - the Serial Monitor should say "left turn!"
9. Cover the middle and right LDR - the Serial Monitor should say "right turn!"
10. Basically just make sure your LDRs are producing dynamic behaviour when you cover them up
11. Unplug the USB cable from the Arduino
12. Plug in the lead acid 12V battery and turn on the Arduino - the little onboard light should glow
13. Now repeat steps 8 and 9 - the stepper motors should turn

You can use any technology you want for setting up a web socket. I wanted the easiest, fastest and cheapest option, so I chose to use the hosted solution provided by Pusher.com which was free.

1. Go to https://pusher.com/websockets
2. Register for a free account
3. Confirm your email address by clicking the link in the email they send to you
4. Setup your channel:
5.  Name your app anything
6.  Choose the cluster closest to your physical location
7.  Choose 'Vanilla JS' as your frontend
8.  Click the 'Create app' button
9.  The 'Getting Started' page will show demo code to get going. This is similar (but not the same) as what appears in the file pusher.php (step 4 below - can wait until then to do this step). Have a look at the Javascript include part (looks like this: <script src="https://js.pusher.com/7.0/pusher.min.js"></script>) near the top, and make sure the URL is the same as what is in the file pusher.php. Maybe the version number, currently 7.0, has gone up since the time of publishing this.
10.  Go to 'App Settings'. Click the toggle next to 'Enable client events' to turn on this feature.
11.  Go to 'App Keys'. Write down the key, secret and cluster. These will go into the top lines of pusher.php (step 4).

You are welcome to install your own socket system or use another provider, but you will need to make changes to the pusher.php file and know your way around Javascript.

1. Get some web space

Ok so you will now be building a website for the bot, or bots (if you are making a fleet!). This website displays the list of bots that are online and can be joined. This website also powers the web page that you will open up in Chrome on your smartphone that powers the interface on the device level. You will need a live domain name (e.g. yourgreatproject.com) connected to hosting space.

This hosting space 1) must allow PHP scripts to be run, 2) must offer MySQL databases, and 3) must have an SSL certificate.

Most hosts these days have LetsEncrypt installed, which is a free method of producing an SSL certificate. The video conferencing software Jitsi Meet will not run on non-SSL domain names (addresses starting with http://), only SSL-secured domains will work (addresses starting with https:// - the little green lock icon).

2. Setup the database

Depending on what software your hosting provider provides to you, you should be able to login to an admin backend and start making changes to your hosting space. My hosting space uses cPanel. You should be able to find a tool called 'MySQL Databases' or similar MySQL admin tool. With this tool, you will need to:

1. Create a MySQL database
2. Create a MySQL user
3. Connect the MySQL user to the MySQL database
4. Write down the details of the database and the user, including password. These will go into the top lines of functions.php.

Now you will need to open up the MySQL console, or open up the MySQL console tool called phpMyAdmin. You will need to run the below command to create the required database table:

CREATE TABLE `bot_list` (
 `id` int(11) NOT NULL AUTO_INCREMENT,
 `hashmini` varchar(10) NOT NULL,
 `name` varchar(20) NOT NULL,
 `last_ping` datetime DEFAULT NULL,
 `session_id` int(11) DEFAULT NULL,
 PRIMARY KEY (`id`)
 ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=latin1;

You should see that a new MySQL table called 'bot_list' has been created. Great!

3. Upload the PHP files

Now you need to upload the PHP files attached below to the default / public_html folder in your hosting space. If you are using a cPanel backend, the tool to do this is called 'File Manager'.

1. Save the TXT files below to your computer
2. Rename each one to have a .php extension instead of a .txt extension. Instructables does not support .php file types so I had to rename them to .txt to upload (annoying).
3. Edit pusher.php in a basic text editor / code IDE like Atom (https://atom.io). Add in your key, secret and cluster from point 7 on step 3 above. Click 'Save'.
4. Edit functions.php in a basic text editor like above. Add in your database user username and password, and then the database name. You might also want to set your timezone by editing the $gmt_offset variable. Mine was +2 hours on the Greenwich Meridian (GMT).
5. Upload all of the files (the ones you edited and the rest) to the default directory / home directory / public_html directory.
6. Remember to set the file permissions to 755 (this is also called CHMODing) so they can execute / run.
7. On your computer/laptop, if you visit your domain name now (earlier we used the example domain of yourgreatproject.com) and load up the bot-web.php file (e.g. https://yourgreatproject.com/bot-web.php) then you should see a screen that says "Bot list - no bots found". Once you complete the next step and add your first robot, this page will automatically update to show the list of bots available to join. It refreshes every 10 seconds automatically to display the bot list.
8. On your smartphone (the one you are using on your robot), open up the Chrome internet browser, and load up the bot-phone.php file (e.g. https://yourgreatproject.com/bot-phone.php). The screen should ask you to name the robot. Enter a name, or enter a name for the place the robot is going to exist, like "Warehouse #1" for example, or "T1000". It can be anything. Click 'Save'. The page should now display "Waiting for visitor".

4. Smartphone setup

In Chrome, you should be able to select 'Desktop site' as a checkbox in the browser settings. Making the browser force desktop view will stop Jitsi Meet from prompting you repeatedly to enable microphone and webcam, and download the Jitsi Meet app. You do not need to download the Jitsi Meet app - just run it via your browser by clicking the 'Launch in web' button.

On the "Waiting for visitor" page there is also a button for making the webpage go full screen. Click this to make it go full screen and hide the Chrome toolbar / menu at the top. You will need a fully blackened screen to put your LDR sensor over so that one sensor doesn't detect the toolbar, with the result being a bot that takes up ballet and twirls endlessly ;)

Ok we are almost done!

On your computer, the list of bots at /bot-web.php should now display your bot name, and the status should say 'ONLINE'. You can now click the 'join' button. This will open up a new page. Read the pop-ups. You will need to allow Jitsi Meet to enable your microphone and webcam.

On your smartphone, the status page at /bot-phone.php after a few seconds should change to a video-conferencing layout, with three black blocks on the right hand side. The status bar at the top should show 'connecting', 'connected' and then 'ready'. You will need to allow Jitsi Meet to enable your microphone and webcam. You should not have to allow again when you connect in future - it should remember your choice.

After a few seconds of settling, from your computer, you should be able to see out of your smartphone. From your smartphone, you should be able to see out of your computer. It is a live video-conference. You can also talk and hear. If this works, you know the video-conferencing via Jitsi Meet is working. Great news!

On your computer, try clicking the driving buttons. On your smartphone, the black blocks on the right-hand side should light up white temporarily, based on which button you push. If this works, you know the socket you setup with Pusher is working - rad!

Now you can place the LDR sensor over the right-hand side of your smartphone screen. Make sure that each LDR sees only one block, with a nice wide berth on either side. No other light except light from the screen should be reaching the LDRs. Fix the LDR sensor with the elastic band so it stays in place nicely.

If you haven't already, stick your smartphone onto the vertical wall on your robot with Prestick / Blu Tack. Now you can turn on the lead acid 12v battery on your robot, and it will come alive! You should be able to drive it around from your computer, see, talk and hear :D

Well done, you just made a telepresence robot!

> If the driver on the website / computer leaves and does not close the browser properly, the bot may get stuck showing an empty video conference. You can just click the little 'x' in the top menu bar to cancel the session and go back to the 'Waiting for visitor" page.

There are lots of potential improvements to make. Here are a few I thought of:

1. Make the bot taller so you can see people eye-to-eye instead of seeing their ankles. This will require a bigger base and a pole to hold the smartphone higher.
2. Make the software support holding down the Forward button, so it is easier to do long continuous forward runs.
3. Make the software handle dropped connections automatically.
4. ADVANCED: Make the base return to a charging dock when battery is low.

Have fun and please share your replications!