Home Assistant Communicator for Those With Disabilities

Many people with disabilities could greatly benefit from the functionality of modern home assistants such as Amazon Echo or Google Home. However, for those who lack the ability to speak, very few options exist to control these devices - our design provides a DIY solution.

In the design of our Home Assistant Communicator, we were particularly motivated by a member of our local community who lacks the ability of speech and fine motor control, but desires to have more independence, in particular with respect to controlling her music. For example, due to her condition, she was unable to pause, play, or control the volume of her music. We designed a device consisting of an assistive switch tray, an Arduino based MP3 player, and a speaker and nicknamed the device T.O.A.D. for Taped Original Assistive Device. T.O.A.D. has a series of buttons located on the tray, which when pressed play customizable recordings of commands for the home assistant. Thus, T.O.A.D. can assist those with motor and speech impairments by allowing access to the various functions of a home assistant.

While we developed the design specifically to communicate with a home assistant, the device functions as an adaptive switch controlled MP3 player which can be programmed with an Arduino to play any pre-recorded MP3 tracks, including entire songs.

Note: In the original design there are eight buttons.

1. Home assistant (one)
2. Arduino IDE coding software
3. Computer (one)
4. 3D printing filament
5. 3D printer (preferably with a base plate at least 360mm by 320mm)
6. Tinkercad account; Tinkercad linked here: https://www.tinkercad.com
7. Tinkercad Designs required and linked below:
8. Button base (one for each button): https://www.tinkercad.com/things/9Y1LovcW9rK-project-toad-button-design
9. Button (one for each button): https://www.tinkercad.com/things/7JiQbGsOg9i-project-toad-buttons
10. Frog tray (one): https://www.tinkercad.com/things/7glhpusg6xU-project-toad-frog-plate
11. Speaker enclosure (one): https://www.thingiverse.com/thing:4731920/files
12. Circuit Enclosure (one): https://www.tinkercad.com/things/apQ581AU8QL-project-toad-electronics-enclosure/edit
13. Wires (red, black, and other colors)
14. VGA cables
15. VGA header linked here: Amazon.com: Pc Accessories - Connectors Pro 10 Pairs D-Sub HD15 High Density Solder Male and DB HD15 Female Connector 3-Row, 20-Pack (10 Male + 10 Female) : Electronics
16. Wire Strippers (or scissors) (one)
17. Tape and pen for labeling wires (optional)
18. Switches (one for each button) linked here: https://www.digikey.ca/short/pq1dvb
19. Micro SD card (two)
20. Auxiliary cord (one)
21. PCB Board (one) linked here: ELEGOO 32 Pcs Double Sided PCB Board Prototype Kit for DIY Soldering with 5 Sizes Compatible with Arduino Kits: Amazon.com: Industrial & Scientific
22. Electronic Parts linked and listed below:
23. Sparkfun MP3 Player Shield (one): https://www.digikey.com/en/products/detail/sparkfun-electronics/DEV-12660/5824154?msclkid=c0d5548102aa1b3f4afd369b662725f2
24. For our project we ordered all the speaker parts at Amazon.com: DROK Mini Ampli Board 6V to 18V Portable Stereo Amplifier Digital Amp Module Audio Amplify Dual Channel 15W+15W with 12V/2A Switch Power Supply Adapter Voltage Regulator and Two Speakers : Electronics, but if it is not available, you can order the parts separately below:
25. Amplifier Board (one): TDA7297 Dual Channel A Version 2 15W Stereo Digital Audio Amplifier Module Electronic Amplifier Board 9V-15V: Amazon.com: Industrial & Scientific
26. Power cable (one): F1TP DC 12V 2A Power Supply Adapter with On Off Switch, 12Volt 2amp Wall Charger for LED Light Strip, DVR/NVR/CCTV Camera, Led Tape Lights, Digital Photo Frame, Router, Yamaha Keyboard : Electronics (amazon.com)
27. Speaker (one): GRS 3FR-4 Full Range 3" Speaker Driver 4 Ohm : Electronics (amazon.com)
28. Arduino Uno (one): Arduino Uno REV3 [A000066] : Electronics (amazon.com)
29. Screws linked below:
30. Screws (one for every button) at https://www.mcmaster.com/90190A108/
31. M2 screws (the amount is dependent upon tray design)
32. Bolts (one for every button) linked here: https://www.mcmaster.com/95462a029
33. Bolt Screw (one for every button) linked here: https://www.mcmaster.com/92865A537/
34. Corresponding screwdrivers and wrenches
35. Soldering Iron
36. Solder

The Assistive Tray consists of the tray and the assistive buttons. The current step focuses on the design of the overall tray, while the next step will focus on the buttons.

1. The Tinkercad model of our tray is provided here (https://www.tinkercad.com/things/7glhpusg6xU-project-toad-frog-plate) and contains a black base piece and a green top in the shape of a frog.
2. To make your own tray design, you will find the hole pattern for the assistive switches to the left of our Tinkercad model. Each individual hole pattern must be at least 40mm from all edges or other hole patterns. When designing your own tray, the top piece needs to have a total height of 29.6mm with a hollow section in the middle with a depth of 24.6mm and a solid top section of 5mm as shown in the image.
3. Next, adjust the hole at the top of the tray to fit your VGA connector or cable.
4. Export your design and print it.

1. For this step, you must print all the button base pieces (https://www.tinkercad.com/things/9Y1LovcW9rK-project-toad-button-design which are a slight modification of the MMC60 Switch from Makers Making Change. (We removed 1 mm from the bottom and 4 from the top of the original switch base shown in the Figure in order to fit into the tray)
2. Once all the pieces are printed follow the assembly directions listed at https://makersmakingchange.com/project/mmc60-switch/ with the exception of the assistive tray replacing the base top and replacing the mono jack and mono cables with two electric wires (one red (positive) and one black (negative)), going out of the mono jack hole. Once the base is assembled, attach the base to the assistive tray with screws as shown in the picture.
3. Next, go to the buttons Tinkercad file (https://www.tinkercad.com/things/7JiQbGsOg9i-project-toad-buttons), where you will find all eight buttons that we used in our design and one extra button which you can use for your own customizations. Our buttons used symbols, colors, and textures to allow for easy differentiation.
4. Then, print as many button tops as your design requires–our design uses eight.

In this step we wire our eight buttons to a VGA cable/connector to later connect to our Arduino based controller.

1. Attach all of the negative (black) wires from the switches to a single connection point (we used a PCB board; however, you can also use a split connector, etc.) and then solder them to the VGA cable/connector.

2.Next, solder each positive wire (red in our enclosure, different colors in our wiring diagram for ease of differentiation) to an individual wire on the VGA cable (example of wires pictured above).

3.Once the wires are soldered, cover the electrical connections with shrink wrap or electrical tape.

4. After the wiring is complete, you can close the Assistive Tray (we used 12mm long M2 bolts).

Note: The wiring diagram above shows the wiring from the switches to the Arduino.

In this step we will discuss the assembly of the speaker used for our Home Assistant Communicator

1. First, purchase the speaker (GRS 3FR-4 Full Range 3" Speaker Driver 4 Ohm : Electronics (amazon.com)), amplifier board (TDA7297 15W+15W Digital Amplifier Module, 30W Audio Power 12V DC Mini Stereo Amp Amplify Dual Channel Power Stereo Compatible with DIY Sound Amplify System Component Car Vehicle Auto Computer Speaker : Electronics (amazon.com)), and power connection cable (Photo Frame, Router, Yamaha Keyboard : Electronics (amazon.com)).
2. Then, print the speaker enclosure files and follow the assembly instructions found at https://www.thingiverse.com/thing:4731920/files.
3. Attach speaker wires, they will be connected to the amplifier board in the Electronic Enclosure Step.

1. Connect the VGA Cable connections (from the assistive tray in Step 3) to the Arduino with MP3 shield. We connected the eight switch wires to analog pins A0-A5 and digital pins 5 and 10 and the negative wire to ground. We used a PCB board so that we did not have to solder the VGA connector directly to the MP3 shield.
2. For our project we used the enclosure linked here: https://www.tinkercad.com/things/apQ581AU8QL-project-toad-electronics-enclosure/edit. (Note: Depending on your specific amplifier board and VGA cable/connectors, you might have to alter the sizing of the enclosure and the hole pattern for the amplifier board.) Print the enclosure.
3. Then, place all the electronic parts in the corresponding locations in the enclosure as shown in the picture.
4. Next, plug the auxiliary cord into the amplifier board and Sparkfun MP3 Player
5. Finally, place the lid onto the enclosure and screw it in with M2 bolts (we used 12 mm bolts).

1. Create the code for the Sparkfun MP3 Player by following the steps linked here: https://learn.sparkfun.com/tutorials/mp3-player-s... In this process you’ll need to create audio tracks. We used the free text-to-speech program linked here: https://learn.sparkfun.com/tutorials/mp3-player-s.... Note that in this process you’ll have to give your tracks names (we simply called our tracks track001, track002...track008). These will be the names you use when you are coding the Arduino Uno, so you may want to write down their names and corresponding sounds. You will need to remember these.
2. Once you’ve created and named your tracks, upload them to a micro SD card and insert it into the Sparkfun MP3 Player.
3. Next, create the code for the Arduino Uno. For this you must first install the Arduino IDE coding software. Once installed, if you have the same number of tracks with the same names, you can use our code - provided in the next step. Otherwise, you can use our code as an example to create your own code.
4. Then, upload the code to the Arduino. Your Home Assistant Communicator is now ready to use.

/**
 * \file MP3ButtonPlayer.ino
 * \This is a modified version pf an MP3 shield sketch originally written by Michael P. Flaga
 *
 * This sketch utilizes the analog and digital input pins to select 8 different MP3 tracks 
 * Tracks must be labeled track001.mp3 through track008.mp3 

 * \note Use this example uses the bounce2 library to provide debouncing fuctions. Advocated by Arduino's website at http://playground.arduino.cc/code/bounce
 */

// libraries
#include <SPI.h>
#include <SdFat.h>
#include <SFEMP3Shield.h>
#include <Bounce2.h> 

/**
 * The following code defines the 6 analog and 2 digitial pins on the Arduino board used for the 8 adaptive switches. 
 */
#define M_0  A0
#define M_1  A1
#define M_2  A2
#define M_3  A3
#define M_4  A4
#define M_5  A5
#define M_6  10
#define M_7  5
/**
 * \breif Macro for the Debounce Period [milliseconds]
 */
#define BUTTON_DEBOUNCE_PERIOD 20 //ms

/**
 * \brief Object instancing the SdFat library.
 *
 * principal object for handling all SdCard functions.
 */
SdFat sd;

/**
 * \brief Object instancing the SFEMP3Shield library.
 *
 * principal object for handling all the attributes, members and functions for the library.
 */
SFEMP3Shield MP3player;

/**
 * \brief Object instancing the Next Button.
 */
Bounce m_0  = Bounce();
Bounce m_1  = Bounce();
Bounce m_2  = Bounce();
Bounce m_3  = Bounce();
Bounce m_4  = Bounce();
Bounce m_5  = Bounce();
Bounce m_6  = Bounce();
Bounce m_7  = Bounce();
/**
 * \brief Index of the current track playing.
 *
 * Value indicates current playing track, used to populate "x" for playing the 
 * filename of "track00x.mp3" for track000.mp3 through track254.mp3
 */
int8_t current_track = 0;

//------------------------------------------------------------------------------
/**
 * \brief Setup the Arduino Chip's feature for our use.
 *
 * After Arduino's kernel has booted initialize basic features for this
 * application, such as Serial port and MP3player objects with .begin.
 */
void setup() {
  Serial.begin(115200);

  pinMode(M_0, INPUT_PULLUP);
  pinMode(M_1, INPUT_PULLUP);
  pinMode(M_2, INPUT_PULLUP);
  pinMode(M_3, INPUT_PULLUP);
  pinMode(M_4, INPUT_PULLUP);
  pinMode(M_5, INPUT_PULLUP);
  pinMode(M_6, INPUT_PULLUP);
  pinMode(M_7, INPUT_PULLUP);

  m_0.attach(M_0);
  m_0.interval(BUTTON_DEBOUNCE_PERIOD);
  m_1.attach(M_1);
  m_1.interval(BUTTON_DEBOUNCE_PERIOD);
  m_2.attach(M_2);
  m_2.interval(BUTTON_DEBOUNCE_PERIOD);
  m_3.attach(M_3);
  m_3.interval(BUTTON_DEBOUNCE_PERIOD);
  m_4.attach(M_4);
  m_4.interval(BUTTON_DEBOUNCE_PERIOD);
  m_5.attach(M_5);
  m_5.interval(BUTTON_DEBOUNCE_PERIOD);
  m_6.attach(M_6);
  m_6.interval(BUTTON_DEBOUNCE_PERIOD);
  m_7.attach(M_7);
  m_7.interval(BUTTON_DEBOUNCE_PERIOD);

  if(!sd.begin(9, SPI_HALF_SPEED)) sd.initErrorHalt();
  if (!sd.chdir("/")) sd.errorHalt("sd.chdir");

  MP3player.begin();
  MP3player.setVolume(10,10);
  
  Serial.println(F("Looking for Buttons to be depressed..."));
}


//------------------------------------------------------------------------------
/**
 * \brief Main Loop the Arduino Chip
 *
 * This is called at the end of Arduino kernel's main loop before recycling.
 * And is where the user's is executed.
 *
 * \note If the means of refilling is not interrupt based then the
 * MP3player object is serviced with the availaible function.
 */
void loop() {

// Below is only needed if not interrupt driven. Safe to remove if not using.
#if defined(USE_MP3_REFILL_MEANS) \
    && ( (USE_MP3_REFILL_MEANS == USE_MP3_SimpleTimer) \
    ||   (USE_MP3_REFILL_MEANS == USE_MP3_Polled)      )

  MP3player.available();
#endif

//Serial.println("No Button");
  if (m_0.update()) {
    if (m_0.read() == LOW)	{
      Serial.print(F("B_PLAY pressed, Start Playing Track # 1"));
       MP3player.playTrack(1);
       
    }
  }

  if (m_1.update()) {
    if (m_1.read() == LOW)	{
      Serial.print(F("B_STOP pressed, Stopping Track # 2"));
       MP3player.playTrack(2);
    }
  }

  if (m_2.update()) {
    if (m_2.read() == LOW)	{
      Serial.print(F("B_NEXT pressed, Start Playing Next Track # 3"));
      MP3player.playTrack(3);
    }
  }
if (m_3.update()) {
    if (m_3.read() == LOW)  {
      Serial.print(F("B_NEXT pressed, Start Playing Next Track # 4"));
      MP3player.playTrack(4);
    }
  }
  if (m_4.update()) {
    if (m_4.read() == LOW)  {
      Serial.print(F("B_NEXT pressed, Start Playing Next Track # 5"));
      MP3player.playTrack(5);
    }
  }
  if (m_5.update()) {
    if (m_5.read() == LOW)  {
      Serial.print(F("B_NEXT pressed, Start Playing Next Track # 6"));
      MP3player.playTrack(6);
    }
  }
  if (m_6.update()) {
    if (m_6.read() == LOW)  {
      Serial.print(F("B_NEXT pressed, Start Playing Next Track # 7"));
      MP3player.playTrack(7);
    }
  }
  
  if (m_7.update()) {
    if (m_7.read() == LOW)  {
      Serial.print(F("B_NEXT pressed, Start Playing Next Track # 8"));
      MP3player.playTrack(8);
    }
  }

}

1. Once you’ve designed all of the individual pieces shown in the steps above, you are ready to begin the final assembly. First, gather all of the previously constructed pieces together.
2. Next, plug the VGA cables into the VGA Headers–one end into the assistive tray and the other into the electronic enclosure.
3. Then, plug-in the two power cables–one attached to the Arduino Uno and the other to the amplifier board.
4. If everything has been built correctly, then the device should be ready to use and should appear similar to the picture above. To use, simply press the button near a home assistant, and it should communicate with it as demonstrated in the video.