Smart Toybox: Kids Will Enjoy Cleaning-up the Toys!

Smart Toybox is the world's first smart toy box that solves problem most parents have - toys all over the house - by making cleaning-up toys a fun game kids will enjoy playing. Additionally, Smart Toybox is Wi-Fi connected, which allows easy and fun customization using provided mobile application, but also other fun and educational games, such as throwing a ball and learning what is left and what is right using multiple connected Smart Toyboxes.

In this instructable we are presenting our prototype that can be made fairly easily with parts widely available.

To build Smart Toybox hardware you will need:

1. TI CC3200 Launchpad. It is available directly from TI ($30) or distributors: http://www.ti.com/tool/cc3200-launchxl

2. TI CC3200 Audio Booster ($29): http://www.ti.com/tool/cc3200audboost

3. Load cell amplifier module based on HX711 such as this one ($10): https://www.sparkfun.com/products/13879

4. Load cell bar. We used 20Kg, but depending on your box it can be lower such as this one (10Kg, $7): https://www.sparkfun.com/products/13329

If you search a bit you can find HX711 module with 20Kg sensor bar for a few bucks ($3 for both) on AliExpress: https://www.aliexpress.com/wholesale?SearchText=hx711+load+cell

5. Wires. For example, you can use these ($2): https://www.sparkfun.com/products/12796. These could be just plugged into existing connector but it is much more durable if you use female connector bar and soldered wires. Sensors come with four wires that have to be soldiered either to 4-pin connector bar or directly to Hx711 board.

6. One or two speakers. These can be speakers from an old TV or laptop or something like this: https://www.reichelt.de/Miniature-Speakers/LSM-S20K/3/index.html?ACTION=3&LA=2&ARTICLE=145866&GROUPID=6559&artnr=LSM-S20K

Audio booster features 3.5mm headphone and line out so you can also use both active or passive stock speakers with this connector. Take a note that TI TPA2012D2 audio amplifier can create output power of about 700mW per channel into 8Ω with 3.3V supply, so act accordingly. If you want more power on output you can install two speakers as this amplifier is stereo. Software is designed to work with both mono, stereo or advanced software mixed mono, so it is up to you to decide or experiment with speaker configuration. Please keep in mind that speakers should be placed on outer edge of the box so the sound can be properly heard.

6. Toy box and one or two wooden, metal or plastic plates with several screws (take a look at your sensor specs, usually it is 2xM4 and 2xM5) and rubber bands to keep things in place. Please read "Basic Principles" chapter to understand what exactly is needed in your case.

7. Power for the electronics. Although this system can be powered with AA or AAA batteries, we used Micro-USB rechargeable power bank for mobile phones that are quite cheap these days and can be found in many flavors, sizes, capacities... You can also power Smart Toybox using power outlet with mobile charger, but it would require outside cable that restricts its usage, and, as it is kid's product, it is not a good idea, because that could be dangerous.

8. (optional) You can make front panel with a button and dual color LED so you can control your Smart Toybox from outside and have light notification of status. For this you will need some breadboard or protoboard, a few resistors, button or micro switch and an LED.

Total budget should be somewhere around $80.

I will not bore you with in-depth explanation how sensors and electronic things work as you can find a lot of information on the Internet. I will just give you overview of basic principles that are important for building the box.

Smart Toybox works by constantly measuring weight inside the box. 20Kg weight sensor bar is capable of detecting change in mass in fraction of about 1 gram, which is more than enough for toys. It will detect even light plastic balls, which is fantastic.

To measure the weight, sensor needs to be attached to two plates - bottom one and upper one where toys should be placed. Although you can use hard bottom of the toy box as the bottom plate, we recommend adding another one and attaching it to the bottom of the box, as it will be easier to debug and make changes. Upper plate should cover whole inner surface of the box with some free space left between the plate and the box sides, so it can "work" and bend the sensor as toys are put inside. This space should not be too large to allow toys to fall through it or to be stuck in between plate and toy box side. In our case 2mm was sufficient.

Picture shows side sectional view of sensor with plates. As you can see, plate should not lay on whole surface of the sensor. This is because forces that bend sensor should come from the nuts, so take a note that a bolt, piece of wood or plastic should separate plate and the sensor on both upper and bottom plates.

Sensors have stickers placed on them with arrow pointing downward, but, in our experience, this indication is not always correct. To address this, we are calculating the corrections in software during calibration. Also, calibration process is necessary as these sensors are analog (with readings +- specific value), they slightly differ between themselves and plates you use will definitely differ, so we need to find out orientation, tare (zero value), threshold, and calculate 1g value.

Another important fact for weight (and sensor) is that gravity force needs to act on a plate. This means that the bottom plate must be flat to the ground, otherwise measurement will be incorrect. However, kids will move their toy box and put it on a side (or upside-down) often to take toys out. Knowing that gravity is acceleration force, we solved this problem using accelerometer (provided on launchpad board), so when it detects non-flat movement, it will stop the game and scream or laugh. This also means that cc3200 launchpad board should be aligned and fixed flat to the bottom plate.

The first thing is to select toy box that will become Smart Toybox. It could be the toy box that you already have or you can make/buy new one. In this step we will try to point some important topics and give you some ideas from our own experience.

Kids love to play, and while they play with toy box they will:

• put box to its side or upside-down to get the toys from the bottom. Our software will react to such behavior, but what is important is to make sure the box you make will be ready for this, as device made from wood can be quite heavy and it can fall and break the toys or even hurt the kid.
• step inside the box. The box and device should be designed in such a way that when kid steps into the box it will not break and hurt the kid.

We tested several designs that are self-standing, meaning that when the box is standing on the floor empty, it maintains its shape, so it is easy to put toys inside. We started with IKEA circular toy box such as this one. It is cheap but has soft bottom, which makes it harder to fix device inside, but can be overcome with dual layer of wood.

As shown in the second and third picture, we used small pizza plates as upper and bottom sensor plates, as it was cheap, wooden (so easy to work with) and available locally.

Our artist friends Dragana Dodig-Pajković and Tanja Đokić made for us a toy box that looks like a (magic) hat. It is made from harder cloth material and has special "pocket" on the bottom to put the device. Sonja Perić also gave it a try with magic hat made from cardboard.

We also tried fully wooden box, it was really great, but it was too heavy to carry around...

Finally, Branislav Aćimović created for us two variants of rectangular collapsible boxes that have hard bottom and an easy way to access and attach device. Rectangular shape is easier to attach front panel (button, LED and speakers). Also, we printed great animal vector images on the sides that we grabbed from Kenney (http://kenney.nl/). We were very happy with these designs and we used them to test and demonstrate our product with kids. More importantly, both kids and parents were also very pleased.

To accommodate new shape and size we had to create new sensor plates. We used HDF panel which is cheap and easy to find. We've cut it in appropriate size and just screwed it to the circular upper panel used with IKEA toy storage. As you can see in the picture, with this multi layer approach it is easy to adapt to desired shape and size.

Later, we also designed new rectangular plates, which was more durable, you can see it on picture beside red box.

And finally, it is also possible to 3D print the plates. These also don’t necessarily have to be in appropriate size, but can be used as base or carrier for full-size plate.

The first step is to prepare TI CC3200 Launchpad board and software framework. You should install UniFlash tool available on TI's site according to instructions that came with your board. If you want to play with the source code, you will also need CCS along with toolchain and SDK.

Check jumpers on your TI Launchpad board for on-board accelerometer — J2, J3, J4 (SDA, SCL, INT) should be short. J4 (INT) is not short by default but luckily TI provides two spare jumpers in the package. There is also jumper placed on the first two pins of P1 connector when board arrives — you should remove it and use it for flash operation. When SOP2 jumper (J15) is in place board will be set to flash mode, waiting for new image to be flashed. So, you should place it when flashing new software to the board. You can read more about SOP jumpers in the manual provided by TI. Connect speakers to your audio booster board, stack the booster board to the CC3200 Launchpad board (watch for the white arrow on both boards that determine first pin on P1 connector) and finally connect HX711 load cell amplifier module and the sensor according to the provided schematic.

If you decide to build front panel, here are pictures how to do it using mini protoboard or using breadboard. It has only one button/micro-switch and dual-color LED (green-red). Note that you can also use SW2 micro-switch on Launchpad as a button.

Build your plates and screw load sensor to them. Think of how to fix the boards and power bank to the plates. Remember that the Launchpad board must be parallel to the plate and properly fixed. Measure twice before you cut and drill. Be patient.

Here are few tips from our experience:

• Bottom side of the bottom plate have to be parallel to the ground. We made a dent so that nut's head doesn't make a bump. The other approach would be to make four legs but this really depends on the box you've chosen.
• Take a look at the picture and you will notice that we used some metal thingies. These are wood-to-metal nut adapter nailed to the wooden surface. If you want to use them, make sure that the hole you drill is under perfect 90 degree angle using drill stand.
• To fix the boards we nailed rubber hemispheres used in plumbing as these happen to be great bumpers. We used rubber bands to hold boards fixed in place.

Grab our software from the GitHub. If you want to play with the software source code, use TI's Code Composer Studio (CCS) provided with the Launchpad board following the steps you'll find in README.md:
https://github.com/nsumrak/Smart-Toybox-Embedded-TI

If you don't want to bother with compilation, we’ve provided zip containing only necessary binary files:

https://github.com/nsumrak/Smart-Toybox-Embedded-TI/releases/download/v1.0/Smart-Toybox-Embedded.zip

Remember to check SOP2 jumper on the board before you flash the software. Than use UniFlash tool to transfer required files to the flash memory before you run application for the first time. Also remember to remove SOP2 before running the application.

Sensor calibration

Every sensor is different and your plate design matters, so you must calibrate software for the sensor. We designed software to automatically enter calibration on the first run, or, to be precise, whenever sensor characteristics are not found in settings.ini file on the board. Plates should be attached to the sensor and device should be placed on flat surface. Make sure there are minimal vibrations during calibration procedure. Prepare predefined load – by default we’ve set it to 1.5l water in plastic bottle. If you want to use any other weight, you can, but you will have to recompile the software. Exact weight is used to calculate digital value of 1g and (currently) it is not used for any critical operation by the software. So, you don't have to be really precise, just find something with weight of approximately 1.5Kg and everything will be fine.

The calibration procedure will start as soon as you power up the board. It will measure empty tray for 8 seconds to find tare (zero load) and weight threshold. Then it will wait for you to put predefined load on the plate. After 3 seconds of stabilization, software will measure load for another 8 seconds and calculate sensor orientation and 1g digital value.

Before you run software for the first time it is advisable to monitor what is going on during calibration. You will need to open serial console. You can use PuTTY (http://www.putty.org/) – select "Serial", check Windows Device Manager for serial port number, select speed of 115200. Run the application, and when instructed, place the load on the upper plate. Note the values, save them to settings.ini, it could come useful later if you want to flash new images, but you don’t want to perform calibration again.

Once calibration is over, you should do some test play with your new Smart Toybox by running software, briefly pressing the button and putting items you have on your desk on the upper plate. Also, it would be advisable to connect it to your Wi-Fi network before you merge device with the box. Check Configuring Wi-Fi step.

Controlling the Box with Button

You can use a SW2 micro-swtich on the Launchpad board or the button on the front panel, if you made one to control toy box’s basic functions.

Since there is only one button on the device, different functions are mapped with different duration of holding the button. This can sometimes be bothersome, so we configured LED to blink each second you hold the button, so you can count LED blinks instead of seconds. For example, if you want to hold the button for 2-3 seconds, you should press the button and count 3 LED blinks before releasing it.

Picture shows the table of all button functions with hold duration and LED blinks.

Kids get bored with same routine very quickly. So, we thought ways to make Smart Toybox exciting for a long time.

Themes

Theme is a collection of sounds. These sounds are 16-bit mono WavPack (http://wavpack.com) compressed samples, organized in "music" playlists, which define order in which sounds will be played, silence periods and repetitions. You can imagine sounds to be tones of some instruments, then these playlists would allow playing of music themes. Or if you imagine sounds to be words, playlists allow connecting them into sentences, saving very sparse space on embedded flash memory. Software is designed to play two independent music streams.

Each "music" has "type" property that defines mood. When Smart Toybox software needs to express its mood, it will play one of the music playlists of specific mood type, either selecting it randomly or consecutively among all music playlists of specific type. Picture shows currently defined types.

Finally, all these data is organized in one single “.STB” file. We’ve made this decision because flash memory on Launchpad board is only 1MB, including firmware and network configuration files, and implemented file system is highly inefficient. Our idea was that mobile application could be used to easily create and share themes. But we needed theme files immediately, before we develop those functionalities, so we created helper application for Windows called "Smart Toybox Theme Packer". Unfortunately, even today this application is still the only way to create Smart Toybox theme. You can download executable here:

https://github.com/nsumrak/Smart-Toybox-ThemePacker/releases/download/v1.0/ThemePacker.zip

Or you can download whole source code and play with it:

https://github.com/nsumrak/Smart-Toybox-ThemePacker

Mobile App

We imagined mobile app as a tool for parents and older siblings to join the play, not just a “controller” for the toy box. The main function is recording new sounds, changing them to be funny and sending them to the box, upon which toy box would reproduce that sound (kids get sooo excited when they hear box is calling them by name). Beyond that, the idea that was left undeveloped was to create and share with friends Smart Toybox themes and games.

We developed our mobile app using Cocos-2dx (in C++) as a cross-platform solution. However, each platform requires some specific code for sound recording and network discovery service, and we developed this functionality only for Android version as we didn't have budget to grab mac+iOs device needed for development. Today it is a proof of concept demo, and you can use it to set up Wi-Fi for Smart Toybox, discover boxes in local network, record sounds, make them funny and send them to the box for playback. You can download Android .apk file (just don’t forget to check security settings on your Android device to allow installation of applications from unknown sources) here:

https://github.com/nsumrak/Smart-Toybox-MobileApp/releases/download/v1.0.1/Smart-Toybox-MobileApp.apk

Or you can clone full source code repository here:

https://github.com/nsumrak/Smart-Toybox-MobileApp

And who knows, you might want to contribute to app development by implementing UI for theme packing on mobile application…

For enabling communication between mobile device and the toy box, devices would first have to be connected through Wi-Fi. This can be done in two ways:

1. By connecting both Smart Toybox and mobile device to the same Wi-Fi network. You can configure your toy box device for this in two ways – by setting the SSID and password of your network directly in the code and recompiling the code (this process will not be covered here, please refer to the Readme file provided with the source code for detailed instructions), and by using “smart config” feature:

• Turn on the toy box. Wait few seconds until the led turns on and off. Press the button for 5 seconds (6 led blinks) to put the device in the smart config mode.
• You will use your mobile device to send info about your Wi-Fi network to the toy box, so make sure that your mobile device is connected to the desired Wi-Fi network.
• Start Smart Toybox app and use the plus in the upper right corner of the screen. In the popup dialog make sure that the SSID field is the same as the name of your Wi-Fi network. If it is not, recheck that your mobile device is connected to it.
• Enter password for your Wi-Fi network in the Key field.
• Press Start button in the popup.
• Now your toy box will try to connect to the Wi-FI network. If it succeeds, it will restart and its led will turn green for 3 seconds. Once this happens, you can press stop in the popup dialog and restart the app. It should now find the toy box and show you the Start instead of Demo button in the lower right corner. Press it to start using the app.
• If something went wrong, restart both the toy box and the app and try again. To restart the toy box, hold its button for 4 seconds (5 led blinks).

2. If the toy box device doesn’t have Wi-Fi network configured, or it fails to connect to it, it will enter Access Point (AP) mode. You can then connect directly to its AP with mobile device:

• Restart the toy box and wait for the led to turn yellow for 3 seconds. This is indicator that your toy box is in AP mode. If you see green instead, the toy box is connected to some Wi-Fi network. If led flashes yellow, network is in unknown state. Try restarting or use factory reset.
• Go to the Wi-Fi settings on your mobile device and find network named “mysimplelink”. Connect to it.
• Start Smart Toybox mobile app, and wait for the app to find the toy box. Once it is found, press “Start” to start using the app. Your mobile device and toy box are now connected.

Smart Toybox company was founded by Nebojša Sumrak, Jelena Petra Marković and Vladimir Perić, with a vision to enrich people's lives by making boring procedures and learning - fun. To achieve that vision, our idea was to use connected technology along with psychology of gamification implemented in software. But, after two years of constant under-funding and different personal desires of founders, we decided to close the company and to publish our work as open-source, giving a chance for our product to live its own life, and for people to build, enjoy and further develop Smart Toybox.

This instructable is about Smart Toybox prototype. We used it to showcase our technology and won three innovation awards. But we didn’t stop there, we designed our own electronic board with new platform, new software and the plastic casing, all built with flexibility in mind, as kid's toy box was the first product from several we planed. However, that is the story for some other occasion…

We hope you will enjoy and have a lot of fun while both building and playing with Smart Toybox!