Hand Tracking With Media Pipe and Arduino to Control the Color of an RGB LED

by owen_the_greenbean in Circuits > Arduino

145 Views, 4 Favorites, 0 Comments

Hand Tracking With Media Pipe and Arduino to Control the Color of an RGB LED

Screenshot 2024-12-23 at 3.38.43 PM.png
I can control color with my hand? | ARDUINO PROJECT | MEDIA PIPE AI HAND TRACKING IN PYTHON

Lets create a project that uses hand tracking to change the color of an LED. This is project will explore using Python and the Media Pipe library to track the position of the hand. After get the position we can then communicate with the Arduino through a wire, also know as serial communication! After the Arduino receives the data from the computer doing the hand tracking we can change the color of the LED.

Supplies

IMG_2191.jpg
  1. A laptop that can connect to an Arduino using a serial cable
  2. An Arduino of any type, in this case I am using an Arduino Nano
  3. Serial cables, 4 are need one for powering the LED and 3 for communicating the color
  4. An RGB 4 pin LED
  5. Breadboard
  6. Serial Cable (The cable must be able to transfer data)

How It Works

ArduinoNano-ezgif.com-video-to-gif-converter.gif

For code and additional file details refer to my github: https://github.com/OwenTheGreenBean/HandTrackingColor

Assemble the Arduino Circut

circuit_image.png

We will be connecting the RGB LED according to the diagram. These pins can be changed as long as you update the pins in the Arduino code.

const int redPin = 9; // Connect to red pin of RGB LED
const int greenPin = 10; // Connect to green pin of RGB LED
const int bluePin = 11; // Connect to blue pin of RGB LED

Upload Arduino Code

Screenshot 2024-12-24 at 4.06.27 PM.png

Connect the cable from the Arduino to the computer, make sure that you have selected the correct port. Note the name of this port for later when we will be modifying the Python code. Using the Arduino IDE we can upload the following code to the Arduino by using the upload button. The Arduino file is attached below

Downloads

handtoled.ino

Python Code

We will now setup the Python Code. To do this download the Python file that is attached and then make sure the librarys are installed. We can do this with the commands.

pip install opencv-python
pip install mediapipe
pip install pyserial

Now that serial port we got from Arduino IDE comes to use now. For the Python program to successfully communicate with the Arduino the correct port must be selected. We will change this line in code to match the serial port we got from the Arduino IDE. Only changing the text inside the ' '.

ser = serial.Serial('/dev/cu.usbserial-210', 9600)

Also! Im am using my laptop that has a built in webcam if yours does not and you are using a USB webcam you can change the 0 to a 1. (0 Being laptop camera and 1 being external camera)

videoCap = cv2.VideoCapture(0)

Downloads

main.py

Run

  1. Make sure the Arduino is connected to the correct port
  2. Close the serial monitor in the Arduino IDE or just exit Arduino IDE application
  3. Navigate to the directory that the Python file is stored in and then using the command
python main.py
  1. You will then have a window pop up that will be tracking your hand and controlling the LED!