Project 30：TinyML-Based Magic Wand

Do you want a smart magic wand to light up and start a fan with a wrist-flick? Traditional hardware control with buttons and code is cumbersome and rigid. This TinyML smart magic wand based on UNIHIKER K10 trains gesture models via Edge Impulse and combines TinyML technology. It can recognize dynamic gestures locally in real-time without internet, precisely controlling RGB lights, fans and other peripherals for contactless control.

Project Objectives

Knowledge Objectives

1.Master model training on the Edge Impulse platform;

2.Understand the concept and application of TinyML;

3.Learn about time series prediction;

4.Master the training and deployment of time series prediction models.

Practical Objective

Make a magic wand with UNIHIKER K10 to control devices like lights and fans by waving it. Choose distinct waving gestures for recognition accuracy. Draw a triangle (Gesture 1), a rectangle (Gesture 2), a heart shape (Gesture 3), or wave UNIHIKER K10 horizontally (Gesture 4) to control the RGB light's on/off and the fan's start/stop.

Materials List

Preparations

Hardware

STEP 1: Connect the DC motor fan to the P0 pin of the UNIHIKER K10 using a white connecting wire.

STEP 2: Save the images from the materials folder to the TF card, and then insert the TF card into the UNIHIKER K10. Connect the UNIHIKER K10 to the computer with a USB cable.

Software

Download and Setup of Arduino IDE 1.8.19

Download and install Arduino IDE software for subsequent model deployment and uploading. The specific download link is as follows: https://www.unihiker.com/wiki/K10/GettingStarted/gettingstarted_arduinoide/

Note: UNIHIKER K10 currently only supports Arduino IDE 1.8.19 or below. We will add support for Arduino IDE V2.x in subsequent releases.

After installation, we need to install UNIHIKER SDK in the Arduino IDE. The specific steps are as follows:

STEP1:open Arduino IDE, go to "File→Preference" in Arduino IDE, and set "Compiler warnings" to none.

STEP 2: add SDK URL.Copy and paste the URL in the box, then click OK to save.

SDK URL Link: https://downloadcd.dfrobot.com.cn/UNIHIKER/package_unihiker_index.json

①③②Copy and paste the link.①③②Copy and paste the link.

STEP3:open "Tools→Board→Boards Manager"，complete the installation of the UNIHIKER K10.

STEP4:In“Tools”, set the board to UNIHIKER K10.

Note: For more instructions on using the Unihiker K10 via the Arduino IDE, please refer to the official wiki: https://www.unihiker.com/wiki/K10/GettingStarted/gettingstarted_arduinoide/.

Hands-on Practice

Next, we'll use UNIHIKER K10 and Edge Impulse platform to train a gesture recognition model. Then, by combining it with RGB lights and fan of the UNIHIKER K10, we'll finish making the magic wand.

Task 1: Connect the UNIHIKER K10 to the AI Platform

In this task, we'll download and install the software, then connect UNIHIKER K10 to the Edge Impulse model training platform via PowerShell.

Task 2: Collect Gesture Data

In this task, we'll upload gesture data to the Edge Impulse platform and finish model training.

Task 3: Make the Magic Wand

In this task, we'll deploy the trained model to the UNIHIKER K10 and finish making the magic wand.

Task 1: Connect the UNIHIKER K10 to the AI Platform

Preparation

Before connecting UNIHIKER K10 to Edge Impulse AI model training platform, we need to install and register the following three programs:

1.Go to the Edge Impulse platform (website: https://edgeimpulse.com/) and complete user registration.

2.Download Python 3 (link: https://www.python.org/).

3.Download Node.js V14 or later (link: https://nodejs.org/en).

Tool Installation

After downloading and installing, open PowerShell as administrator. The specific steps are as follows:

After opening it, enter "npm install-g edge-impulse-cli --force" to install the Edge Impulse CLI tool. The specific steps are as follows:

Task 2: Collect Gesture Data

Establish Connection

To collect gestures with UNIHIKER K10, enable its gesture sensing and transmit the sensed gestures to the Edge Impulse platform for model training.

First, open Arduino IDE software. Copy the code from“code1.txt” in“Project 30” folder into the code area. Then, click“Upload”to upload the data forwarding code to UNIHIKER K10.

Next, open PowerShell and enter the command “edge-impulse-data-forwarder --frequency 100” to forward data from the UNIHIKER K10 to the Edge Impulse platform.

After program runs, modify the code. Enter your Edge Impulse account in the corresponding field and name the UNIHIKER K10. Finally, assign unique variable names to the five output variables in the code. Here, I name them k, x, y, z, v.

After running successfully, the connection between the UNIHIKER K10 and the Edge Impulse platform is established.

Data Collection

Next, open and log in to the Edge Impulse platform to collect gesture data.

STEP 1: Go to“Data acquisition”section, enter a label for the motion sensor, select 2000 ms as the sample duration, and start collecting data.

After clicking “Start sampling”, you have 2 seconds to wave the UNIHIKER K10 to record the gesture.

In this project, we use four gestures to control the RGB light and fan:

Drawing a triangle (Gesture 1) → RGB light on

Drawing a rectangle (Gesture 2) → RGB light off

Drawing a heart shape (Gesture 3) → Fan open

Waving the UNIHIKER K10 horizontally (Gesture 4) → Fan close

Note: Before sampling, make sure the same type of gesture corresponds to the same label.

During sampling, ensure enough gesture data is collected for “Training” and “Test” sets. The Edge Impulse platform uses the Training set to train the AI model for gesture recognition and the Test set to verify model performance by testing recognition accuracy with unseen gestures.

STEP 2: After collecting data, we can go to“Create impulse”to set the size and frequency of the eigenvalue acquisition window.

The eigenvalues can be generated as illustrated in the following figure.

Task 3: Make the Magic Wand

Model Training

After data collection, enter the “Classifier” to train the model. Set the training cycles (e.g., 100), select the model type (float32 is slightly larger but more accurate), and click “Save&Train” to start training.

If you are not satisfied with the training results, you can re-collect gesture data and train the model again until you are satisfied.

If not satisfied with training results, re-collect gesture data and retrain the model until satisfied.

Model Deployment

Next, export and deploy the model: Click“Deployment”, select Arduino Library and TensorFlow Lite, then start building. Finally, download the Arduino library.

Download “conv&depthwise.zip” file from the "Project 30" folder, copy it to the libraries folder of Arduino IDE 1.8.19, and extract it.

Then copy conv.cpp and depthwise_conv.cpp to the library path below:src → edge-impulse-sdk → tensorflow → lite → micro → kernels

Next, download “RMT&DERobot_NeoPixel_lib.zip” file from "Project 30" folder, copy it to the libraries folder of Arduino IDE 1.8.19 and extract it.

Then, after files are copied, open Arduino IDE, copy the code from“code2.txt” in the "Project 30" folder to the code area and upload it to the board.

Note: Due to the inclusion of the Edge Impulse model, this code will take a very long time to compile, approximately 40 minutes.

Finally, to make the magic wand clearer and more attractive, display a corresponding image on the screen as a prompt when a gesture is drawn and the corresponding action is triggered. Store the images in the TF card in advance, with the specific paths as follows:

Run the Code

After uploading is complete, you can power on the UNIHIKER K10 and control the RGB light and fan by drawing gestures.

Knowledge Base

Next, let's learn and summarize the knowledge used in this lesson.

Concept and Applications of TinyML

TinyML (Tiny Machine Learning) is a technology that deploys machine learning models to small resource-constrained devices such as microcontrollers and sensors.

Different from traditional AI that requires powerful computing resources, TinyML enables lightweight models to run efficiently on edge devices through model compression, quantization, and optimization, achieving local real-time processing and significantly reducing latency and power consumption. In our project, TinyML technology allows Unihiker K10 to quickly identify birds without internet connection, truly realizing "edge intelligence".

Time Series Forecasting

Time series forecasting refers to taking a sequence of time-ordered data (such as daily temperature, monthly sales, and hourly passenger flow) as the research object. By analyzing the hidden variation patterns and trends in these historical data, it estimates the corresponding values or development trends in the future, so as to predict subsequent conditions in advance.

Challenge Task

After building the basic Tiny ML-based smart magic wand that controls RGB lights and fans with four gestures, add at least two distinct dynamic gestures. Re-collect data, train and deploy the model on the Edge Impulse platform, and assign independent control functions to the new gestures to create a more complete Tiny ML intelligent interactive work.