Have you wanted a magical box to read your emotions? Imagine it can detect facial expressions, set off fireworks when you're happy, play music when angry, and send reminders when tired, being your caring companion. In this lesson, we'll create a YOLO-based emotion recognition mood box with the UNIHIKER K10 and explore human facial expression recognition via computer vision!
Project Objectives
Knowledge Objectives
1.Understand facial keypoints;
2.Understand the methods of facial expression detection;
3.Use K10 network image streaming;
4.Use YOLO for face keypoints.
Practical Objective
Build a mood box with UNIHIKER K10. When a human face enters the device's sensing range, it shows expressions of happiness, anger, or tiredness. The UNIHIKER K10's intelligent recognition system captures and identifies these expressions in real-time and responds. For anger, it plays soothing music; for happiness, it displays fireworks; for tiredness, it triggers an alarm.
Materials List
Preparations
Hardware
Insert the TF card into the UNIHIKER K10's corresponding slot, and then connect the UNIHIKER K10 to the computer via a USB cable.
Software
Open Mind+, switch to“Upload Mode”, connect the UNIHIKER K10 as shown in the figure below, and load the UNIHIKER K10 library.
Hands-on Practice
Next, we'll use the UNIHIKER K10's expression recognition function to capture and recognize facial expressions in real time, implement corresponding functions, and create a mood box.
Task 1: Network Configuration & Network Camera Startup
First, we'll connect the UNIHIKER K10 to the network, enable the network camera function, and establish a visual communication channel with the computer to transmit video data.
Task 2: Vision Detection & Result Upload
Then, use the UNIHIKER K10 camera to capture facial expressions in real-time and transmit them to the computer. Meanwhile, connect the UNIHIKER K10 to the MQTT, and the computer will upload the detection results to the SIoT .
Task 3: Making the Mood Box
Last, the UNIHIKER K10 will remotely get inference results from SIoT and give corresponding prompts for different expressions to finish the production of the mood box.
Task 1: Network Configuration & Network Camera Startup
Code
Network Configuration
First, retrieve and add the Wi-Fi module from the Module in the "Extensions". The specific steps are as follows:
Next, we can complete the core code follow the main logic:connect to the network→wait for successful connection→output IP address, as shown below:
Note: Connect UNIHIKER K10 and computer to the same Wi-Fi network.
Network Camera Startup
After connecting the UNIHIKER K10 to the network, search for and add the “K10webcam”from "Module" in extensions. The specific steps are as follows:
After loading, follow the main logic to display the UNIHIKER K10 camera and enable the network camera to transmit the feed to the LAN. Then, any LAN computer can access it at any time and use existing computer vision libraries for image recognition.
To enable the network camera, use“Webcam (open)”block from UNIHIKER K10 Webcam, as shown below.
The complete reading code is shown below:
Run the Code
Click the“Upload”button, wait for upload completion, then open the Serial Monitor. You'll see the IP address of the UNIHIKER K10, as shown below.
Open the browser and enter IP/stream to view the camera feed. For example, enter 10.119.123.130/stream (the IP in the above figure) in the browser. The corresponding screen is as follows:
Code Review
Category | Blocks | Function |
 |  | It toggles the network camera's on/off status and works with other blocks to display the UNIHIKER K10 onboard camera's footage on a computer in real time. |
Task 2: Vision Detection & Result Upload
Code
Next, connect the UNIHIKER K10 to the MQTT platform. Create a SIoT topic to store results, and use a computer for facial keypoint detection to analyze expressions based on these keypoints. Then send the detection results to SIoT for the UNIHIKER K10 to do subsequent processing.
Computer Environment Preparation
On the computer side, download the Windows version of SIoT V2 and extract the SIoT_V2_Win.zip file.
After extraction, double-click start SIoT.bat to launch the new version of SIoT. Once launched, a small black command window pops up to start the server. Don't close it while the project is running, and note your computer's IP address.
Double-click "start SIoT.bat"You can view the local IP address.Double-click "start SIoT.bat"You can view the local IP address.
Note: For the download and installation of SIoT_V2, please refer to https://github.com/liliang9693/siotv2/releases。
Next, open the Mind+ software to complete the MQTT connection.
First, search for and add the“MQTT”from the“Module” in Extensions. The specific steps are as follows:
Then complete the code in this sequence: Initialize parameter settings→Initiate connection→Judge connection status & give prompt. Reference code is as follows:
Combine Task 1 code, omit prompt section after successful connection, and the complete reading code is as follows:
Then, to recognize and process key facial information, we need to install the Python dependency libraries for this project. The specific method is as follows:
STEP 1: Open a new Mind+ window or your own python IDE. If you choose Mind+ (V1.8 and above) , you can select and enter“Code”under “Python”.
STEP 2: Click“Library Management”to open the library installation page. In “PIP Mode”, run the following commands in sequence to install six libraries, such as the mediapipe and ultralytics libraries.
Code
STEP 1: Create a Topic
In the computer's web browser, enter local IP address:8080.For example, enter 192.168.35.1:8080 in the browser, as shown below:
STEP 2: Enter the username "siot" and password "dfrobot" to log in to the SIoT IoT platform.
STEP 3: Create a new topic "emo" for data transmission.
STEP 4: Write code for facial expression detection. The code calculates key- point positions to implement expression recognition.
First, create a new Python file named "visiondetect" in”File in the project” in Mind+'s right sidebar.
Next, we use the YOLOv8 model to detect facial key points. Based on the mouth opening degree, eye aspect ratio, eye closure duration, and eyebrow position in the image, we send commands to the "emo" topic on the SIoT platform. Copy the code from the attached visiondetect.py file to the visiondetect.py file you created in Mind+.
The code uses a multimodal feature extraction+rule-based threshold decision method for emotion recognition, mainly combining facial features.
In most cases, people open their mouths when happy. So, when the lip distance exceeds 30, it's classified as happy.
When the mouth is tightly closed (upper-lower lip distance<8), eyebrows are lowered and their vertical distance<30, it's classified as angry.
If you close your eyes for over one second, you might doze off, and you'll be detected as very tired then.
Since we need to get the camera feed from UNIHIKER K10, replace the IP address on line 27 of the code with your UNIHIKER K10 IP address. The specific operation is as follows.
Run the Code
Click button and wait for loading to finish. Position your face fully in the camera window of the UNIHIKER K10 and make different facial expressions in front of the camera. The recognized expression results will be updated in real-time in the terminal window, as shown below.
Task 3: Making the Mood Box
In this task, we'll implement functions on the UNIHIKER K10 with SIoT's facial expression data. Specifically, play soothing music for an angry expression, show fireworks on the screen for a happy one, and trigger the buzzer for an alert when fatigue is detected.
First, use a card reader and a computer to store required music and image files into the TF card. Then, insert TF card into the corresponding slot of the UNIHIKER K10. The storage locations of the material files are as follows:
After the addition is complete, subscribe to the corresponding SIoT topic in the main program of the code for Task 2, as shown below:
Next, use"When (“topic/a”)received (MQTT message)" block in MQTT to enable the smart terminal to process control blocks from "siot/emo", as shown below.
Finally, use "If() then()" block for conditional judgment and execute corresponding functions to complete Mood Box making. The specific logic is as follows:
The corresponding code is as follows:
The complete code for reading is as follows:
Run the Code
Click the“Upload”buttonand wait for upload completion. Make corresponding expressions in front of the UNIHIKER K10 camera, then observe the screen changes and music playback of the UNIHIKER K10 board.
Knowledge Base
Next, let's learn and summarize the hardware knowledge used in this lesson.
Facial Keypoints
Facial Keypoint Model is a computer vision model. It detects and locates key face feature points (e.g., eyes, nose, mouth, eyebrow contours) via algorithms, outputs their coordinates in the image, and quantitatively describes face shape and pose.
These detection models are widely used in daily life and form the basis for the following use cases.
Expression Analysis and Recognition
By tracking positional changes of facial key points, the model identifies user emotions such as happiness, anger and fatigue. It can be used in intelligent hardware interaction (e.g.,expression control on UNIHIKER K10), affective computing, user experience optimization, etc.
Human-Computer Interaction (HCI)
Dynamic changes of facial key points including eye blinking, mouth opening and head rotation are captured as interaction commands. This supports contactless control (e.g., waking devices by blinking) and assistive equipment for people with disabilities.
Medical and Healthcare Applications
Long-term variations in key points, such as facial muscle asymmetry, are analyzed to assist in diagnosing neurological disorders (e.g., facial paralysis) and to assess skin aging and sleep quality.
Facial Expression Detection
The core of facial expression detection is to capture facial feature changes via technical means and identify emotion categories (e.g., happiness, anger, sadness, fatigue). It's mainly divided into three types.
Detection based on facial keypoints
Detection based on image texture features
Deep learning-based end-to-end detection
Facial expression detection models are widely used in daily life, such as:
Security & Behavior Monitoring
Automotive safety: Detect driver expressions to judge fatigue or distraction and issue timely alerts.
Human-Computer Interaction & Intelligent Hardware
Capture players’ facial expressions to control in-game character emotions, or adjust game difficulty based on expressions .
Education & Training
Detect students’ in-class expressions, allowing teachers to adjust teaching pace in real time or generate personalized learning reports.
Challenge Task
Based on the YOLO-based expression recognition function of UNIHIKER K10 , add a new expression category, Sadness. Adjust facial keypoint detection threshold rules for accurate recognition and configure an exclusive response effect. Specifically, display a corresponding image on the screen and play soothing music.
