Building a Robot That Follows Your Mouse Cursor

In this instructable I will show you how to build and control a robot capable of drawing and writing on a flat surface, as illustrated in the video above.
To complete this project you will need:

• A computer
• A download of Processing IDE: a simple, free, open-source, program development tool (no installation required).
• An Arduino board : a simple, open-source, micro-controller.
• Two standard size servomotors, I used two Hitech HS-5645MG.
• A heavy duty, quarter-scale servomotor, I used a Power HD-1235MG.
• Some mechanical hardware to build a frame holding the two servos, the two arms and the writing tool. I used Actobotics components and you will find a complete list of pieces I used in the Mechanism sections of this instructable.
• A breadboard and jumper wires for electrical connections.
• An independant DC current source for the servomotors (alternatively, use the one provided by the arduino board).
• A biro and a piece of wire (here I used a piece of bicycle brakes cable).

Basic mechanism

We will use two servomotors, each attached to an articulated arm. Each arm is made of two articulated segments, connected to a servomotor at one end and to the writing tool at the other end. The position of the writing end of the robot arms is completely determined by two angles controlled by the servomotors.

Basic architecture

Below is a quick preview of the setup. The computer will run a processing program which tracks the position of the mouse cursor, and send it over to the arduino board. This program is detailed in the Processing section of this instructable.

The computer is connected to the arduino board through the arduino's USB cable.

The position of the mouse cursor is converted to appropriate angles for the servomotors, this is detailed in the Mathematics section of this instructable.

The arduino is connected to servomotors through simple wires, Arduino controls the servomotors through another program. The wiring and arduino program is detailed in the Arduino section of this instructable.

The mechanical structure holding the servomotors, articulated arms and pen is detailed in the Mechanism section of this instructable.

This section explains how the position of the mouse cursor on the screen of the computer is converted into angles for the servomotors so that the writing end of the articulated arms replicates the movement of the mouse cursor.

It is not necessary to understand this to successfully complete the intructable, as I will provide the pieces of code which implement the conversion of cursor position to servomotor angles.

Analyzing a single servomotor

If we analyze a setup with a single arm and servomotor (figure above):

Call O the origin of our referential. It is placed on the rotation axis of the servomotor.
Call P the point we want to reach on the writing plane, or the pen's position.
Call (x,y) the coordinates of P.
Call a the length of the arms segments. (Note: if all arm segments are of equal length, the calculations are much easier).
Call J the position of the joint where the two arm segments are connected.
Call r the distance OP : r = (x²+y²)^1/2
Call θthe angle between OP and the x axis : θ = atan2(y,x)
In the triangle OJP :
The distance OJ = a
The distance JP = a
Call φ the angle PÔJ : φ = acos( r / (2a) )
Call α the angle which must be fed to the servomotor to reach position P :
α = θ + φ
α = atan2(y,x) + acos( (x²+y²)^1/2 / (2a) )

Note that feeding this angle to a single servomotor will not constrain the pen's position to a point. It will constrain J to a single point but will leave P free to move on a circle of radius a around J. Adding the second servomotor will allow us to constrain P's position more.

Adding the second servomotor

We can repeat the calculations done in the previous paragraph, except this time the point we want to reach P, is seen in the referential of the second servomotor, which is offset from the referential we used earlier.

Call d the distance between the axes of the two servomotors.
The coordinates of P in the referential of the second servomotor are : (x+d,y)
The configuration of the arms of the second servomotor is similar but mirrored, so that we find :
Call α₂ the angle which must be fed to the second servomotor to reach position P :
α₂ = θ₂ - φ₂
α₂ = atan2(y,x+d) - acos( ((x+d)²+y²)^1/2 / (2a) )

Feeding the angles α and α₂ to the two servomotors will constrain P to be at the intersection of two circles = two points. We can restrict the domain of P in such a way that the pen will initially be placed at the topmost of these two points and never have the chance to switch to the other intersection point.
This way we can fully control the position of the pen by feeding two angles to the servomotors.
The images above show the shapes of the reachable domain on the writing plane for different values of arm length (a) and servomotor offset (d).

Get the processing IDE:

Go to the processing web site, navigate to the download section, download the processing Integrated Development Environment and run it (no installation required).

Launch processing, create a new program (also called sketch) from the File menu.

Write a sketch:

Paste the following code in your sketch :

Note : the program will not run correctly until the arduino is connected to the computer, this is explained in the Running everything section of this instructable.

import processing.serial.*;

// The data which was las sent to arduino :
int _LastSent_MouseX;
int _LastSent_MouseY;
int _LastSent_PenUp;

// The most current values of mouse position and pen up/down state :
int _Current_MouseX;
int _Current_MouseY;
boolean _Current_PenUp;

// Properties of the drawing window :
int _WindowWidth;
int _WindowHeight;

// Port for communication with arduino
Serial _SerialPort;

//////////////////////////////////////////////////////
// Setup the Processing Canvas
void setup() {
  _WindowWidth = 1600;
  _WindowHeight = 800;
  size( _WindowWidth, _WindowHeight );
  strokeWeight( 8 );

  // Open the serial port for communication with the Arduino
  // Make sure the COM port is the same as the one used by arduino (Tools > Serial Port menu)
  _SerialPort = new Serial(this, "COM7", 9600);
  _SerialPort.bufferUntil('\n');
}

//////////////////////////////////////////////////////
// Draw the Window on the computer screen
void draw() {
  // Fill canvas grey
  background( 100 );  
  // Set the stroke colour to white
  stroke(255);
  // Draw a circle at the mouse location
  ellipse( _Current_MouseX, _Current_MouseY, 8, 8 );
}

//////////////////////////////////////////////////////
// Updates the most current state of mouse position and sends this information to arduino
void mouseMoved() {  
  _Current_MouseX = mouseX;
  _Current_MouseY = mouseY;
  SendToArduino();
}
// Updates the most current state of pen up/down state and sends this information to arduino
void mousePressed() {
  _Current_PenUp = !_Current_PenUp;
  SendToArduino();
}

//////////////////////////////////////////////////////
void SendToArduino() {
  // it is more performant to send simple ints over the serial port to the arduino board
  // however, it is hard to guarantee that the data is received in the right order on the arduino side
  // to avoid misinterpreting sent data over the serial connection, we use the following convention :
  // X position will be sent as as int of value 0..127
  // Y position will be sent as an int of value 127..255
  // Up/Down state will be sent as an int of value 100 or 200
  int new_MouseX=round(map(_Current_MouseX, 0, _WindowWidth, 0., 127.));
  int new_MouseY=round(map(_Current_MouseY, 0, _WindowHeight, 255., 127.));
  int new_PenUp= _Current_PenUp ? 100 : 200;                  //

  // we are only going to send the data over to the arduino if one of the values changed
  if (new_MouseX!=_LastSent_MouseX || new_MouseY!=_LastSent_MouseY || new_PenUp!=_LastSent_PenUp) {
    _LastSent_MouseX = new_MouseX;
    _LastSent_MouseY = new_MouseY;
    _LastSent_PenUp = new_PenUp;

    _SerialPort.write((byte)new_MouseX);
    _SerialPort.write((byte)new_MouseY);
    _SerialPort.write((byte)new_PenUp);
  }
}

//////////////////////////////////////////////////////
// this is useful to display the messages sent from the arduino board over the serial port
// for debug purposes
void serialEvent(Serial myPort)
{
  String myString = myPort.readStringUntil('\n');
  println(myString);
}

Get the arduino IDE:

Go to the Arduino web site, navigate to the download section, download the arduino Integrated Development Environment and install it. Launch it, create a new program (also called sketch) from the File menu.

Write a sketch:

Paste the following code in your sketch :

#include <Servo.h>
// to activate debug outputs, uncomment the following line:
//#define DEBUG

Servo servoLeft;
Servo servoRight;
Servo servoUp;
const float pi = 3.14159;
float RadianToDegree = 180/pi;
float DegreeToRadian = pi/180;

// the length of the arm segment (dimensions are scaled so that arm segment length is equal to 1.)
const float a = 1.;
// the offset between the two servomotor axes:
const float d = 5.7/14.7;
// to convert incoming bytes to float values:
const float ByteToFloat = 1.0/255.0;
// security margins to avoid exiting the safe drawing zone:
// note that the drawing area has the same x/y ratio here than in the processing sketch
const float thetaMax = acos(d/2. - 1.);
const float YMin = sin(thetaMax) + d;
const float XAmplitude = 2.2;
const float YAmplitude = 1.1;
const float XMin = -0.5*(XAmplitude-d);
const float XMax = d-XMin;
const float YMax = YMin+YAmplitude;
// pen position (dimensions are scaled so that arm segment length is equal to 1.)
float x=0.;
float y=1.2;

// for serial port communication:
int MyDelay=10;
int incomingByteX = 0;
int incomingByteY = 0;
int incomingByteU = 0;

// initialize servomotor positions and serial port
void setup()
{
  Serial.begin(9600);
  servoLeft.attach(9);
  servoRight.attach(10);
  servoUp.attach(6);
  servoLeft.writeMicroseconds(1940);
  servoRight.writeMicroseconds(1095);
  servoUp.writeMicroseconds(1100);
}

// the following method reads the serial inputs, 
// converts incoming data to pen positions
// converts pen positions to servomotor angles
// sends angle values to servomotors
void loop()
{
  // wait for 3 bytes
  if(Serial.available() >= 3) ;  
  {
    incomingByteX = Serial.read();
    incomingByteY = Serial.read();
    incomingByteU = Serial.read();
   // check that incoming data respects the convetions we set in the processing sketch
   // this avoids misinterpreting data that comes unordered
    if((incomingByteU==100 || incomingByteU==200) 
    && incomingByteX>=0 && incomingByteX<=127 
    && incomingByteY>=127 && incomingByteY<=255)
    {
      #ifdef DEBUG
      Serial.print("SERIAL X : ");Serial.print(incomingByteX);
      Serial.print(" ||  SERIAL Y : ");Serial.print(incomingByteY);
      Serial.print(" ||  SERIAL U : ");Serial.print(incomingByteU);Serial.println();
      #endif
  
      x = incomingByteX*ByteToFloat*2*(XMax-XMin) + XMin;
      y = (incomingByteY*ByteToFloat - .5)*2*(YMax-YMin) + YMin;
     
      // convert (x,y) values to servomotor angles
      float ThetaLeft = GetThetaLeft(x,y);
      float ThetaRight = GetThetaRight(x,y);
      
      // send the positions to the servomotors
      SetServoLeftToTheta(ThetaLeft);
      SetServoRightToTheta(ThetaRight);
      ServoUp(incomingByteU==100);
    }
  }
    delay(MyDelay);
}

// the following methods convert (x,y) positions to servomotor angles
float GetThetaLeft(float x, float y){
  float R = hypot (x,y);
  float Theta = atan2(y,x);
  float Phi = acos(R/2.);
  float Out = RadianToDegree*(Theta + Phi);
  return Out;
}
float GetThetaRight(float x, float y){
  x = (x-d);
  float R = hypot (x,y);
  float Theta = atan2(y,x);
  float Phi = acos(R/2.);
  float Out = RadianToDegree*(Theta - Phi);
  return Out;
}
// the following methods set angular positions on the two arm servomotors
float SetServoLeftToTheta(float ThetaLeft){
  // calibration for left servo Hitech-5645MG
  // theta      |     uS
  //---------------------------
  //                 2200
  //   90            1940
  //  180            1020
  //                  750
  float mS = 1940. + (ThetaLeft-90.)*(1020.-1940.)/90.;
  servoLeft.writeMicroseconds(round(mS));
}
float SetServoRightToTheta(float ThetaRight){
  // calibration for left servo Hitech-5645MG
  // theta      |     uS
  //---------------------------
  //                 2200
  //    0            1995
  //   90            1095
  //                  750
  float mS = 1995. + (ThetaRight)*(1095.-1995.)/90.;
  servoRight.writeMicroseconds(round(mS));
}
// the following method brings the lifting servo to pen-up or pen-down position
void ServoUp(boolean PenUp){
  if(PenUp){ servoUp.writeMicroseconds(1000);  }
  else     { servoUp.writeMicroseconds(1200);  }
}

Here is a list of all the Actobotics parts used in this project:

• 4x 6.16" (17 holes) aluminum beams
• 3x 3.85 (11 holes) aluminum beams
• 1x 1.54" (5 holes) aluminum beam
• 1x 12" Aluminum channel
• 1x 3.75" Aluminum channel
• 2x 1.50" Aluminum channel
• 2x 90° Single angle channel Bracket
• 4x Flat single channel bracket
• 2x Flat triple channel bracket
• 4x 90° Hub mount C
• 2x Standard servo plate D
• 1x Beam Attachment block
• 4x 4mm Bore set screw hub
• 3x 4mm Bore clamping hub
• 6x 90° Dual side mount E
• 2x 4mm (100mm length) Precision shafting
• 1x 4mm (300mm length) Precision shafting
• 2x Standard Hitec ServoBlocks (or equivalent hubs, hub mounts and ball bearing plates)
• Actobotics Hardware Pack A (screws and nuts)

The servos I used are :

• Hitech HS-5645MG
• Power HD-1235MG

I also used :

• A biro
• A piece of cable (bycicle break cable)

Here we are putting together elements on the main channel, the base of our robot.

For this step, we use :

• 1x 12" Aluminum channel
• 1x 4mm (300mm length) Precision shafting
• 4x 90° Hub mount C
• 2x Flat single channel bracket
• 4x 4mm Bore set screw hub

Attach the each hub mount to a a bore set screw hub :

Slide these elements on the long shaft :

Attach the hub mounts to the main channel, using the flat single channel brackets to offset them a little (this will give room for the rotation motion around the shafting to move the pen up and down) :

Your main channel now looks like the pictures below. The two central hub mounts and hubs are free to translate on the shaft :

Here we are assembling a case-like assembly around the pen-up (power HD) servo. This will later be used to attach the servo on the main channel.

For this step, we use :

• The power HD servo

• 1xFlat single channel bracket

The result should look like this :

Here we are assembling a case-like assembly around the pen-up (power HD) servo. This will later be used to attach the servo on the main channel.

For this step, we use :

• 1x 3.85 (11 holes) aluminum beams
• 1x Flat single channel bracket
• 2x 90° Dual side mount E

The result should look like this :

Here we are assembling a case-like assembly around the pen-up (power HD) servo. This will later be used to attach the servo on the main channel.

For this step, we use :

• The assembly from the previous step
• 1x Flat single channel bracket
• 2x 90° Dual side mount E

The result should look like this :

Here we are assembling a case-like assembly around the pen-up (power HD) servo. This will later be used to attach the servo on the main channel.

For this step, we use :

• The assembly from the previous step
• 1x 1.50" Aluminum channel

The result should look like this :

Here we are assembling a case-like assembly around the pen-up (power HD) servo. This will later be used to attach the servo on the main channel.

For this step, use the two parts of the pen-up-servo case you just assembled, assemble them together for you giant-size servo to have a snug little case ready to be attached to the writing robot :

Here we are attaching the case-like assembly around the pen-up (power HD) servo on the main channel. This is straight forward : See the images above.

Here we are adding two servo plates on the main channel, they will later hold the two angle servos.

For this step, we use :

• 2x Standard servo plate D
• 1x 3.75" Aluminum channel

Attach the servo plates to the small 3.75" channel like this :

And attach the servo plate to the hub mounts which you left free to rotate/translate on the long shaft on the main 12" channel :

Attach the second servo plate in the same manner. The servo plates + small 3.75" channel should now have a bit of room to rotate due to the fact that you raised the long shaft a little with the flat single channel brackets in the first building step.

Your robot now looks like this :

Here we are attaching two hub mounts on the servo plates we just added.

For this step, we use :

• Standard Hitec ServoBlocks / Hub mounts

The servo plates now look like this :

The servos fit snugly in the servo plates, as pictured above.

For this step, use :

• 2x Hitech HS-5645MG servos

Notice I chose to have the servo axes close together instead of split apart. This impacts the shape of the domain reachable by the pen. See the Mathematics section for more details.

These plates will serve to hold the servo hubs tightly in place.

For this step we use :

• Standard Hitec ServoBlocks / Ball bearing plates

Attach them to the hub mounts like this:

The servo hubs are attached to the rotating servo spline. The robot arms will be attached to these hubs.

For this step use :

• 2x Standard Hitec ServoBlocks / Servo hubs

Fit the servo hubs through the ball bearing plates and attach them to the servo splines :

Here is a side view of the two servo hubs attached to their servos :

Two 90° Single angle channel Bracket are attached to the front of the servo plates. These will later be used to attach two short shafts which will go from the servo mounts to the writing surface, to avoid the pen hitting too hard on the writing surface.

For this step use :

• 2x 90° Single angle channel Bracket

Screw them on the hub mounts like this :

Here is what it looks ilke after the two brackets are screwed on :

This little assembly will be attached to the moving part which makes the pen go up and down. The pen-up servo will pull on it to raise the pen.

For this step use :

• 1x 1.54" (5 holes) aluminum beam
• 2x 6x 90° Dual side mount E
• 1x Beam Attachment block

The result should look like this :

Attach the small assembly you just built to the ball-bearing plates as illustrated above.

Run a cable through the hole of the Beam Attachment block and to the pen-up servo horn, as illustrated above and below :

Your robot with the pen-up mechanism complete now looks like this :

For this step we use :

• 2x 4mm (100mm length) Precision shafting
• 2x 4mm Bore clamping hub

Attach the small 100mm shafts to clamping hubs in this manner (Don't tighten them just yet, you'll need to adjust the length so that they touch the writing surface at the same time as the pen when the pen is raised / lowered) :

Here we'll attach the short shafts and clamping hubs (aka "resting legs") to the 90° Single angle channel Bracket we added earlier, as pictured above.

Once both resting legs are mounted on the robot, it should look like this from the bottom :

And like this from the top :

For this step, use :

• 2x 6.16 (17 holes) aluminum beams

Attach these beams to the servo hubs. Note that the 2nd hole of the beam is in line with the servo's rotation axis. The efficient length of the arm segment is therefore 16 holes (5.6").

One arm is attached :

And now both arms :

Here we are attaching a second arm segment or "forearm" to the arm.

We are using the second hole of the forearm in order for all effective segments to be of the same length (16 holes).

For this we use :

• 1x 6.16" (17 holes) aluminum beams

We are not attaching both forearms just yet, as we will be building the pen attachment system to the other forearm in the next steps.

For this step we will use :

• 2x 3.85" (11 holes) aluminum beams
• 2x Standard Hitec ServoBlocks / Hub mounts
• 1x 4mm Bore clamping hub

Attach a hub mount to the 3.85" aluminum beams like this :

Then attach the clamping hub to the hub mount like this :

And finally, attach the second hub to the opposite side of 3.85" aluminum beams like this :

A standard biro will fit perfectly in our pen holding system !

Here we are attaching the pen holding system to our remaining arm segment, as illustrated above.

We will use :

• 1x 6.16" (17 holes) aluminum beam
• previously built pen holding system

Notice that once against it is the 2nd hole of the arm segment which is aligned with the pen point. Our effective arm segment length is 16 holes for all arm segments.

The two forearm extremities are joined on the axis exactly above the pen point.

Notice the arm segments' over/under configuration.

It is now time to bask in the beauty of our magnificent robot and perform a global checkup.

In this step we will connect the servomotors to the arduino board. There are different kind of hardware to make these connections. I used a breadboard and jumper wires.

I used an external DC power source to power the servos. While it is possible to power them directly from the arduino board's USB connection, the USB power source offers limited current outputs. For the servos to move faster and more efficiently, use an external power source. You can use a small DC connector on the breadboard to do so.

The arduino code provided in an earlier section of this tutorial expects the following connections :

• Left servo connected to pin 9 of the arduino.
• Right servo connected to pin 10
• Up-Down servo connected to pin 6

You can see the diagram of the connections above.

It is time to play with the robot ! Above is a longer video showing the robot in action.

This is your final checklist :

1. Connect the arduino to the PC with an USB cable.
2. Compile the arduino sketch and upload it to the arduino board, using the arduino IDE's compile and upload buttons.
3. Check what COM port your arduino is using under the Tools > Serial port menu of the arduino IDE.
4. Check that the processing program is using the same serial port, the relevant line is in the setup()method.
5. Compile the processing program.
6. Make sure the ground ountput of the arduino board is connected to the ground output of your DC power supply if you are using one.
7. Un-connect the forearm extremities and pen-raising system cable before powering the servos to avoid inappropriate and unexpected movements.
8. Check the arm segments rotate easily at their joints.
9. Check the resting legs hit the writing surface at the same time as the biro point, to avoid the biro point taking too big hits when the pen is lowered.
10. Check the arm segments pass easily over the clamping hubs holding the resting legs (if not you can raise the arms a little by inserting a flat metal ring between the arm and the servo hub).
11. Check the servo wires or other electrical wires do not risk getting torn by the movement of the arms.
12. Power, test and calibrate the servo positions individually so that the angles taken by the servos are precisely the ones expected (see diagram in Mathematics section). This can be done either by programming digital servos with this kind of device, or by making adaptations in the arduino code. The relevant methods are SetServoLeftToTheta and SetServoRightToTheta.
13. Launch the processing program from the processing IDE.
14. The robot should now follow the movement of the mouse cursor on the processing window of your computer screen.

Have fun !