How Can I Run Teachable Machine on a Microcontroller?

This compact device operates independently from a computer, allowing users to conveniently experience the fascination of machine learning.

Materials

Microbit (or Arduino, or other boards with serial ports)

Sugar-Camera (a circuit board based on ESP32-S3)

Robotbit expansion board (for IO port conversion and power supply)

A few Lego structural components (for building the framework)

MicroUSB data cable and PH2.0 4PIN connector cable

software

Teachable Machine（Teachable Machine）

MakeCode（https://makecode.microbit.org/）

KittenBot_Teachable Machine（A compact tool designed by Xiaomiao Technology that seamlessly connects microcontrollers with the Teachable Machine platform.）

Teachable Machine is an online tool developed by Google that aims to empower users to train their own machine learning models through simple interactions. It eliminates the need for coding, instead allowing users to gather data through features like cameras, microphones, and image uploads. With this data, users can train their models by providing different labels and examples. Teachable Machine automatically analyzes and learns from these examples, enabling it to recognize and categorize similar data in the future.

This tool can be used for various applications, including image classification, pose recognition, sound recognition, and more. It is especially well-suited for beginners and educational purposes.

Most of the time, online tutorials focus on teaching how to utilize a camera for model training and then recognize the results on a computer. However, for some small projects, we desire to operate independently from a computer. We aim to import the trained model into a microcontroller and control sensors such as displays and servo motors based on the outcome.

To achieve a machine learning object classification system independent of a computer, our goal is to import the trained models from Teachable Machine into a microcontroller with an integrated camera, specifically the ESP32-S3. By establishing a serial connection with the Microbit, we can accomplish the desired outcome.

This compact device operates independently from a computer, allowing users to effortlessly and conveniently experience the fascination of machine learning. It serves as a tangible manifestation of the accomplishments achieved solely through software-based learning in Teachable Machine, enhancing the enjoyment of the learning process.

Please ensure that Sugar-Camera has been flashed with the dedicated firmware for Teachable Machine.

Open this tool

Establish a connection with the serial port of Sugar-Camera. Upon successful connection, the interface will display the image feed.

Follow the instructions shown in the diagram below to create a machine learning project for the microcontroller.

Collect different samples according to the instructions in the diagram below.

Three categories have been recorded here, namely grapebit, LED, and None

The names of the categories can be modified. The recorded images can be individually deleted or further added. Generally, a recognition effectiveness of over 20 images is sufficient.

Click on "train" and await completion of the training process.

Establish a connection with the device

Examine and assess the identification of three objects. In the event of inaccurate recognition, proceed with the collection and input of additional data into the model, followed by training and subsequent testing. Repeat this cycle to ensure continuous improvement.

Upon confirming that the recognition performance of the model meets the requirements, proceed with exporting the model for integration into a microcontroller.

Utilizing our proprietary tool, transmit the compressed ZIP package containing the model to the Sugar-Camera device.

Reconnect once again to obtain the recognition results.

Establish a serial connection between the Microbit and the Sugar-Camera.

Source code of the program:

https://makecode.microbit.org/_PuvHgaWR27PT

The program, as shown in the image, requires you to modify the pins and the identification type name for the Serial. Once you have made these changes, proceed to download the program.

Recognition Test

The exported model images from Teachable Machine's microcontroller project are 96x96 in size and in grayscale, which limits the recognition performance to some extent.

To enhance the recognition rate, it's advisable to select objects with significantly distinct shapes and appearances.

It is hoped that Teachable Machine can gradually expand its capabilities, allowing for larger sample image sizes and the inclusion of various colors. This expansion would greatly improve the recognition effectiveness.