Easy Follow

With this tutorial the first low budget version of an automatic driving carriage can be realized, which can follow a person in front of the robot. The following instructions are divided into the implementation steps of the drive, the housing, the electronics and the software.

The drive of the carriage is realized by a modified vacuum cleaner robot, which is converted so that the motors of the vacuum cleaner robot can be controlled externally. The housing provides space for the required electronic components of the robot and is designed in such a way that the patient's supply devices could be placed on it.

The functional principle is based on the fact that a camera system detects the patient and reports its position. The position is processed on a computer and a movement command is generated for the drive unit. The motors of the drive unit can be controlled via a connected arduino with a motorshield and the robot with the supply devices moves to the desired position.

For this purpose, the source code is started on the laptop, which makes the robot ready to drive and follow.

The process of preparing the vacuum cleaner is explained step by step in 17 pictures. Each picture has an explaining text. So please open the pictures to begin with the process.

The assembly of the electronic parts is described in the following step “Hardware-Electronics”.

Materials &Tools

Materials

- Arduino UnoAdafruit motorshield v2

- Laptop

- Microsoft Kinect

- iRobot Roomba 555

- Assorted wires

- Solder

- Tape

- Small container ( for example bread box)

- 2 small screws with treads for wood or sheet metal

Tools

- Soldering Iron

- Small Flat Head screwdriver

- Small Philips head screwdriver

- Side cutters for cutting wires and removing insulation

- Small saw for cutting the Roomba

Materials and Tools Introduction

In this case, for just demonstrational reasons, cardboard was the chosen as material to build the chassis. For actual usage it can also be worked with other materials such as wood. The dimensions remain the same. For this particular application, device holders for devices Freelox and MyAirvo2 were considered. Due to the modular structure, other devices can also be considered, sizes need to be adjusted for that.

Material

- 2 (cardboard) boxes of the same size 35x35cm with lits

- 1 (cardboard) box size 14 x 21cm without lit (size of Freelox)

- 1 (cardboard) box size 31 x 18,5cm without lit (size of MyArivo2)

- 8 toothpicks

-12 Din-A4-Paper (preferable used scrap paper)

- 1 bamboo rod 120cm

Tools

- scissors

- glue

- duct tape

Step 1: First level

- Cover one box (35x35cm) with paper, for a nicer look

- Box and lit need to be wrapped separately

- Cut holes in the Box

- One at the side on the bottom at the place where the USB-inputs of the Laptop are. (see image)

- Two holes for laptop ventilation. (Here on the bottom and the opposite to the first hole, depending on the Laptop) (see image)

- These holes can later be used for assembly so the lower the better.

Step 2: Second level

- Cover the second box with paper for a nicer look

- Box and lit need to be wrapped separately

- Glue the lit of the first box to the bottom of the second box (see image)

- Cut a holes in the Box

- One hole of approxamitly 3x3cm in the bottom of the bot (later for cable) (see image)

- Two holes of the size of the camera and camerafoot (see image)

Step 3: perforated plate

- punch holes into the lit of the second box using the scissors

- first draw straight lines 2,5cm from the edges and 5cm from themselves (see image)

- punch holes with the scissors where the lines cross

- widen one of the holes on an edge for the bamboo stick to fit

Step 4: Device holders

- Cover the boxes with paper for a nicer look

- if needed add ventilation holes

- punch 4 holes each in the box

- holes need to be a multiple of 5cm apart (see image)

- wrap sellotape around one side of the 8 toothpicks (see image)

- Place the toothpicks in the holes with the sellotape inside the box (see image)

Step 5: Bamboo stick

- cut the bamboo stick in pieces of 20 and 100cm

- sellotape the small piece orthogonally to the bigger piece (see image)

Step 6: Assemble

- Tape the first box to the vacuum cleaner using the input an ventilation holes. If needed add more slits (see image)

- Add the lit and second box combination. (see image: first assembly)

- Add lit with holes

- Add device holders Add bamboo stick and sellotape it inside the second box. (see image: final assembly)

This section contains a step by step instructions on how to assemble the electronics for the EasyFollow. If you plan on building a prototype of your own construction, the general assembly of the electronics can be used for any build, that utilizes the same concept for self driving. The motorshield can be used to drive any compatibe motors, not just those of a repurposed Roomba.

General construction of electronic elements

The electronics part of the assembly consists of 4 main parts:

- MS Kinect Camera

- Laptop

- Arduino Uno v2 + Adafruit motorshield v2.3

- Motors / motor assembly

Connecting the MS Kinect Camera

- Place the Kinect inside the 3rd level of the chassis, so that it looks out of the horizontal opening in the side. The base protudes through the smaler opening below.

- Fix the camera in place using double-sided adhesive tape.

- Split the cable of the Kinect to an USB connection and a 12v connection to a power supply. The USB connection is used to connect the camera to the laptop. The instructions for this can be found at: Wiring an Xbox Kinect for USB

- Thread both cables through the hole in the bottom of level 2 (of the chassis) into level 1 and the USB cable out of level 1 through the laptop opening. The USB cable is then looped back and can later be plugged into the laptop.

Arduino + Motorshield

- Place the motorshield on top of the Arduino using the matching connectors. Depending on the model of motorshield purchased, the connectors on/underneath the motorshield come pre installed or have to be soldered on.

- The instructions for setting up the motorshield can be found at the manufacturers website: Install Headers & Terminals

- Attach the four cables from the motors (two from each motor) and a 12v power supply to the connectors on the motorshield

- Attach a USB cable to the Arduino.

- Thread the power supply cable underneath the vacuum to the side and from the outside throught the laptop opening into level 2.

- Thread the USB cable underneath the vacuum to the side so it can later be plugged into the laptop.

Power supply

- Thread an extension cord with a multi plug through the laptop opening in level 2.

- The power supply for the camera, the motorshield and the laptop charger are pugged into it inside level 2.

- Plug the extension cord into an outlet once the assembly is finished

Download Python

Go to https://www.python.org/downloads/ and download the latest Python Version.

Install Python

Start the downloaded Python installer. Make sure you add Python to Path (second picture). Then proceed with the installation. When the installer is done, make sure to disable the path length limitations (third picture).

Download VS Code and install it

Go to https://code.visualstudio.com/download, download the latest VS Code Version and install it.

Install VS Code Python Package

Navigate in the VS Code enviroment to extension. Search for the Python extension and install it (second picture).

Introduction

We are using a Microsoft Kinect v1 for this project. Its very cheap for its capabilities and easy to get.

Download the SDK

To use the Kinect camera with a windows pc you need to download the SDK (Software Developement Kit):

https://www.microsoft.com/en-us/download/details.aspx?id=40278

Be aware: the SDK version 1.8 is the latest working version for the Kinect v1 camera. Do not try to use the SDK 2.0, it does not support our camera, its for the Kinect v2 camera.

Installation

Install the downloaded SDK.

Introduction

We use Nuitrack to process the datastream of the Kinect camera as we could achieve good person tracking results with it.

Receive license

Go to https://nuitrack.com/#pricing and get the license that fits your personal needs the best. Follow the given download and installation instructions, after purchasing your license.

Install PyNuitrack

We used the PyNuitrack package, to realise the tracking code in python. Follow the instructions to install it on your system:

https://github.com/3DiVi/nuitrack-sdk/blob/master/PythonNuitrack-beta/README.MD

Activate the moving sensor function

The last step is to activate the moving sensor function. It is deactivated by default. When activated, the sensor can process movement of the tracked object and the sensor itself. When not activated the sensor thinks his postion in the room is not changing, leading to tracking failures when the sensor is moved. Follow the instructions given from nuitrack to activate the moving sensor function:

https://github.com/3DiVi/nuitrack-sdk/blob/master/doc/Moving_Sensor.md

Download the code

Download the code files and place them in a folder.

Open the code in VS Code

Open the downloaded Python code in VS Code. Check which Python modules are missing on your system to start the code.
Follow this instructions to install missing packages in Python:

https://docs.python.org/3/installing/index.html

Download the Arduino Code

Download the given Arduino code file

Download the Arduino IDE

Go to https://www.arduino.cc/en/software to download the Arduino IDE and install it.

Check Arduino IDE settings

Connect the Arduino via USB to your Computer and open the Arduino IDE. Make sure to set up the right COM port for the flashing. If you struggle with this step you can find help here: https://support.arduino.cc/hc/en-us/articles/4406856349970-Find-the-port-your-board-is-connected-to

Also make sure that you selected the right Arduino board (see picture). We used an Arduino Uno Rev3.

Flash the code

Open the downloaded code in the arduino IDE. Click on the Upload button (second picture) to flash the code to your Arduino.

In this case "Thingsboard" is used as a IOT Dashboard.

Step Set up Thingsboard

Create an account under: Link

https://thingsboard.cloud/signup

verify your Account via Email and log in the Thingsboard.

It is advised to get familiar with the Thingsboard by working through the “Getting Started” guide: Link

https://thingsboard.io/docs/getting-started-guides/helloworld/

Step Create Devices and Dashboard

Create Device

• Select “Device groups” on the left.
• Create a new Device group by clicking on the “+” symbol
• Enter a Name, eg. “Assistance Trollies”
• Enter the device group and create a new device called “my_device”
• klick on the device and klick on “copy access token” save the token on your PC

Set up Dashboard

Alternative 1:

Upload premade Dashboard - not advised a smooth working cannot be guaranteed and errors my occur.

• klick on “Import Dashboard” and select the premade Dashboard.
• before uploading the file must be converted from a .txt to a .json file, you can do that by changing the ending of the filename.
• The Dashboard should set up.

Alternative 2: Create custom Dashboard - advised

• Create Dashboard
• Select “Dashboard groups” on the left.
• Create a new Dashboard group by clicking on the “+” symbol
• Enter a Name, e.g. “Assistance Dashboards”
• Enter the device group and create a new device called “my_dashboard”
• Enter the created Dashboard and enter the edit mode
• Create an alias calls “device1” to your device, as learned in the “Getting started guide”
• Create a new Basic GPIO control widget, as learned in the “Getting started guide”
• Select “device1” as target device
• Modify the widget:
• Enter the “Advanced” setting
• Delete toggle switch 3
• Name toggle 1,2 to “Start” and “Start Following”
• Set the RPC Timeout to 5000
• Klick on “Apply Changes” of the device
• Klick on “Apply Changes” of the Dashboard
• Create nonmandatory indication LEDs
• As learned in the “Getting stated guide” create a new Control widget Led indicator
• Select your “device1” as the target device
• Go to “Advances” Setting
• Name your LED, e.g. Started
• Under “Retrieve led status value using method” select “Subscribe for timeseries”
• Under “Device attribute/timeseries containing led status value” enter “status”
• Klick on “Apply Changes” of the device
• Klick on “Apply Changes” of the Dashboard
• repeat this process twice under “Device attribute/timeseries containing led status value” ender “in_use” and “user_found”
• The LEDs will indicate:

1. If the trolly stated
2. If the trolly is following
3. If the trolly found a user

• If desired the following Widgets can be created:
• “Vertical bar”: Name: Battery; Datasource: ”device1”; timeseries: “battery” - This bar indicates the battery level of the trolly.
• “Timeseries Line Chart”: Name: Battery; Datasource:” device1”; timeseries: “battery”
• “Timeseries Line Chart”: Name: Battery; Datasource: ”device1”; timeseries: “speed_of_movement_interday”

You are now ready to try your prototyp. Have fun.