Accessible Painting

This interactive art allows you to paint on your laptop without directly having to use your mousepad or keyboard! This was inspired by an activity we did in class when we made accessible buttons. Following this, I wanted to create a way where anyone, including people with immobility, could paint using touch sensors and accelerometer.

Supplies

Raspberry Pi Pico W
Breadboard
3D printing materials (10)
Box Laser Cutting
Copper Foil Tape
2 STEMMA QT 4 pin Cable
7 alligator clips
1 MPR121
1 LIS3DH
1 small, round velcro
1 hook and loop velcro

Laser Cut Box and 3D Printouts

First, you're going to want to print out a box that has 7 holes on the top for the touch sensing buttons and two holes on the side for the cables to connect. The 3d printouts of the buttons should have a blocking layer, so the buttons don't fall through holes in the wooden cut. The buttons also have a hole the a form of a thicker line so that we can connect the mpr121 capcitive touch sensors with copper foil tape and alligator clips. The red 3d printout goes with the accelerometer and should be a little bigger than the LIS3DH block. The cap should match exactly to the bottom so that it doesn't fall. The yellow 3d print is to slide in the hook and loop velcro so that it can wrap around our hand.

Assemble Them

Touch Sensors

Tape the copper tin tape to the 3D printouts.
Attach it to the box

Accelerometer

Add the small round velcro to the inner part of the red 3d print and to the LIS3Dh so that the accelerometer can be detached pretty easily, but doesn't move when we move the parts and give us accurate readings
Attach the yellow 3d print to the bottom of the red facing the same direction of the red 3d printout
Slide in the hook and loop velcro

Code the Painting App

I used Tkinter, a GUI in Python, to code my painting application. You can run this code my typing python paint-project.py in your terminal. Nearly all the code is encoded in class main: and is called in the end for more efficiency:

if __name__ == '__main__':

root = Tk()

main(root)

root.title('Paint App')

root.mainloop()

Inside the class, some of the important parts contain the binding of mouse and keyboard inputs. We need this because our Pico W program is going to send it and the Tkinter is supposed to react to it.

self.c.bind('<B1-Motion>', self.paint) # Holding Left Mouse

self.c.bind('<ButtonRelease-1>', self.reset) # Releasing Left Button Mouse

self.c.bind('<Button-2>', self.clear) # Clicking Middle Button Mouse

self.c.bind('<Button-3>', self.useEraser) # Clicking Right Button Mouse

self.c.bind('<KeyPress-Left>', self.increaseWidth) # Pressing Left Arrow

self.c.bind("<KeyPress-Right>", self.decreaseWidth) # Pressing Right Arrow

self.c.bind("<KeyPress-Up>", self.upColor) # Pressing Up Arrow

self.c.bind("<KeyPress-Down>", self.downColor) # Pressing Down Arrow

All labels in the Tkinter application is used in this syntax:

# Label Colors

Label(self.controls, text='Colors:', font=('times 16')).grid(row=2, column=0)

and the Buttons come like this:

self.decrease_button = Button(self.controls, text='<', command=self.downColor, font=('times 16'), padx=14, pady=15)

self.decrease_button.grid(row=2, column=1)

.grid is used to position the elements and padx=14, pady=15 is used to put more space inside the button.

Downloads

paint-project.py

Code in Your Pico W (code.py)

Inside code.py, we're going to import these libraries:

import board, busio, adafruit_lis3dh, adafruit_mpr121, time, usb_hid

from adafruit_hid.mouse import Mouse

from adafruit_hid.keyboard import Keyboard

from adafruit_hid.keycode import Keycode

from adafruit_debouncer import Button

Now, just a quick overview of the code, we're going to connect the accelerometer and capacity touch sensors to the board. Make sure that you have two i2c variables so that their address don't overlap each other:

# Set up I2C for accelerometer and touchpad

i2c = busio.I2C(board.GP5, board.GP4) # SCL=GP5, SDA=GP4 for Pico W

i2c2 = busio.I2C(board.GP7, board.GP6)

# Initialize the accelerometer

accelerometer = adafruit_lis3dh.LIS3DH_I2C(i2c, address=0x18)

accelerometer.range = adafruit_lis3dh.RANGE_8_G # Set accelerometer range

# Initialize the touchpad

touch_pad = adafruit_mpr121.MPR121(i2c2)

In the code, you're going to find that there is a mapping of the accelerometer to the mouse so that the mouse would move according your movement:

# Track the actual mouse position

current_x, current_y = 0, 0

# Flag for using the accelerometer

use_accelerometer = True

Going inside the while True loop:

while True:

current_x, current_y = 0, 0

# Disable accelerometer if any pad is pressed

use_accelerometer = not any(pad.pressed for pad in pads[:7])

# Read acceleration values only if use_accelerometer is True

if use_accelerometer:

x_acc, y_acc, z_acc = accelerometer.acceleration

# Map the accelerometer values to mouse movement

x_move = int(pow(abs(x_acc), 1.6))

y_move = int(pow(abs(y_acc), 1.6))

# Invert movement direction based on acceleration

if x_acc > 0:

x_move = -x_move

if y_acc < 0:

y_move = -y_move

# Move the mouse based on accelerometer

mouse.move(x_move, y_move)

You will also find the pads have different mouse and keyboard assignments. I decided to connect pads[0] to pen, which is going to paint it so while the pads[0] is pressed, the mouse is going to press instead of click.

pads[0].update()

if pads[0].pressed:

mouse.press(Mouse.LEFT_BUTTON)

if pads[0].released:

mouse.release(Mouse.LEFT_BUTTON)

We're also going to loop around to get through the rest of pads and assign the rest of the pads to either a mouse or keyboard:

for i in range(1, len(pads)):

pads[i].update() # Get the current state of each debounced pad

if pads[1].pressed:

mouse.click(Mouse.MIDDLE_BUTTON)

if pads[2].pressed:

mouse.click(Mouse.RIGHT_BUTTON)

if pads[4].pressed:

kbd.press(Keycode.LEFT_ARROW)

if pads[4].released:

kbd.release(Keycode.LEFT_ARROW)

if pads[3].pressed:

kbd.press(Keycode.RIGHT_ARROW)

if pads[3].released:

kbd.release(Keycode.RIGHT_ARROW)

if pads[5].pressed:

kbd.press(Keycode.UP_ARROW)

if pads[5].released:

kbd.release(Keycode.UP_ARROW)

if pads[6].pressed:

kbd.press(Keycode.DOWN_ARROW)

if pads[6].released:

kbd.release(Keycode.DOWN_ARROW)

# Release the mouse button if any of the pads are released

if any(pad.released for pad in pads[1:7]):

mouse.release(Mouse.LEFT_BUTTON)

This isn't everything in the code, but the most important parts explained. So feel free to download the code and put it inside your Pico W!

Downloads

code.py

Wiring

Connect your touch sensing buttons to the mpr121 to the alligator clips. If you're following the code above, connect pads[0] to the pen button, pads[1] to the eraser button, pads[2] to the clear button, pads[3] to the + button, pads[4] to the - button, pads[5] to the ^ button, and pads[6] to the ⌄ button. Now, using the STEMMA QT, connect the mpr121 to the breadboard with the Pico W. The black wire should go to the GND, red to 3.3V, blue to GP.6, and yellow to GP.7. Connect you lis3dh accelerometer with the other STEMMA QT to the breadboard. The black wiring should go to GND, the red should go to 3.3v, the blue to board.GP4, and the yellow to board.GP5.

Play It

Move your mouse to the top left corner, so that the coordinates don't get mixed up. Use it!