Music Mirror | Huskylens Playground with micro:bit EP01

“Mirror, Mirror, on the wall, who’s the fairest of them all?”I am sure you’ve heard this childhood fairy tale "Snow White and the Seven Dwarfs". The Magic Mirror from the story is an all-time classic and has been on the list for years.

Now, with HUSKYLENS, we can make such a magic mirror on our own. It’s difficult to make the mirror speak like a person, but we can add a buzzer on it to let the mirror play different music to express its attitude. Let’s make a music magic mirror together!

Function Introduction:

In this project, we will learn the face recognition function of HUSKYLENS, use its built-in machine learning technology to distinguish the learned face from other faces, and play different kinds of music. This is the magic mirror.

Think about it, when you point the HUSKYLENS camera at your face, it will play beautiful music that expresses compliments, and when you point at others’ face, it will play fun music, which may get people confused.

Materials:

Micro:bit     https://www.dfrobot.com/product-2125.html

IO Extender for micro:bit V2.0     https://www.dfrobot.com/product-1867.html

HUSKYLENS     https://www.dfrobot.com/product-1922.html

Knowledge Field:

Like other biological characteristics of the human body (fingerprint, iris, etc.), the face is inherent, unique, and not easily copied. It is the first kind of image that has been studied, and also the widely used one in the field of computer vision. This project is making use of the face recognition of HUSKYLENS.

1. What is Face Recognition?

Face recognition is a biological recognition technology for identity recognition based on information about facial features. It uses a camera to collect images and videos containing human faces and automatically detects and tracks the faces, then performs a series of analytical techniques of the detected faces.

Operating Principles:

There are four key steps of face recognition: 

Here is a simple introduction of these four functions:

Face Detection: detect the face positions and select them by frames.

Face Alignment: recognize faces from different angles by locating the features.

Face Encoding: extract face information and make it understandable for a computer.

Face Match: match the face information with the existing database to get a similarity score and give out the matching result. Face recognition is regarded as one of the most difficult courses in the field of biological recognition even artificial intelligence due to the features of the face as biological characteristics.

Face recognition is also considered to be one of the most difficult research topics in the field of biometrics and even artificial intelligence. The difficulty of face recognition is mainly caused by the characteristics of the face as a biological feature.

Similarity: There is not much difference between individuals. All the faces, even the facial organs are similar in structure and shape. Such a feature is beneficial to the use of human face for locating, but unfavorable for distinguishing individuals.

Mutability: Facial appearance is not very stable. Through changing the expression of the face, or observed in different angles, the visual images vary widely. Besides, it is also influenced by the light condition (such as day and night, indoor and outdoor, etc.), cover (such as masks, sunglasses, hair, beard, etc.), age, etc.

Application Scenarios:

Access Control System: Through face recognition, the identity of the person entering into the protected areas such as prisons, detention houses, neighborhoods, schools and houses, can be recognized.

Camera Monitoring System: It can be used for monitoring the crowd in public places such as banks, airports, stadiums, shopping malls, and supermarkets. For example, the monitoring system can prevent the terrorists from boarding the plane.

Online application: Use facial recognition to assist credit card online payment to prevent non-credit card owners from using credit cards or social security fraudulent claims, etc.

Face recognition is now widely used in many fields, such as student attendance system, camera, phone unlocking, all-in-one machine for human identification, etc.

2. Demonstration of HUSKYLENS Face Recognition

Back to the smart access control system, due to the build-in learning ability, HUSKYLENS can recognize faces. It works like a collector for a database, marking the face information that is specified to be input.

How to operate it in detail? Take out your HUSKYLENS and let’s do it once together.

* If you use this camera for the first time, please refer to the product WIKI page for firmware burning and language setting:

https://wiki.dfrobot.com/HUSKYLENS_V1.0_SKU_SEN0305_SEN0336#target_5

Step1. Power on

HUSKYLENS comes with an independent USB power port, connect the USB cable, then you can turn it on.

Step2. Select the “Face Recognition” Function

Dial the “function button” to the left until “face recognition” is displayed at the top of the screen.

Step3. Learn Faces

Point HUSKYLENS at the area with faces on it. When a face is detected, it will be automatically selected by a white frame with the words "Face" on the screen.

Point the “+” symbol at the face that needs to be learned and short press the “learning button”. If the same face is detected by HUSKYLENS, a blue frame with the words "Face: ID1" will be displayed on the screen, which indicates that HUSKYLENS has learned the face before and can recognize it now.

*Long press the “learning button” then let HUSKYLENS learn the face from different angles.

*If there is no “+” symbol on the screen, it means that HUSKYLENS has already learned in the current function. Short press the "learning button", the screen will display "click again to forget". Before the countdown ends, short press the "learning button" again to delete the learned face information.

Project Practicing:

After learning the basic steps of HUSKYLENS, let’s put the music mirror into practice.

The first thing we need to do is to make the camera distinguish faces both learned and unlearned. The second is to add at least two pieces of music to realize playing music. Finally, you can use an small mirror to build the exterior. We will divide this project into two parts.

Task 1: Distinguish Faces

In this step, we first learn how to use the HUSKYLENS camera to recognize and distinguish human faces, determine whether the faces have been learned, and send some feedback.

Task 2: Add the Music

After learning how to distinguish faces and perform feedback functions, we can add more functions on this basis, such as playing different music after recognizing different faces.

Task 1: Distinguish Faces

1. Hardware Connection

2. Program Design

The face discrimination here is that the camera has learned the specified face information in advance. When the camera recognizes the face again, it can determine whether it has been learned. In order to make the display process more straightforward, when a smiling face is displayed on the main control board screen, it means that the specified face is recognized, and a crying face means it is not.

Before programming, design the program through the following 3 steps.

Step1: Mind+ software settings

Open Mind+ (Version 1.6.2 or above), switch to "Offline mode", click "extension", click "micro:bit" under “Board", and click "HUSKYLENS AI Camera" under "Sensor".

Step2: Command Learning

Here are the instructions mainly used.

① Initialize only once between the beginning of the main program and looping executions. You can select I2C or Soft-serial, and no need to change I2C address. Please note that the “Output protocol”of your HuskyLens sensor should be set to be consistent with the program, otherwise, data cannot be read.

② You can switch to other algorithms freely, but please note that you can run only one algorithms at each time, and it takes some time to switch algorithms.

③ The main controller requests HuskyLens to store data in the “Result” once(stored in the memory variable of the main board, and once requested, it will refresh the data in the memory once), then the data can be obtained from the “Result”. The latest data can be got from the “Result”only when this module is called.

④ Check whether there is frame or arrow in the screen from the requested “Result”, including the learned(id>0) and unlearned, and returns 1 if there is one or more.

⑤ Check whether the IDx has been learned from the requested “Result”.

⑥ Check if the IDx requested from the “Result”is in the screen. The frame refers to the algorithm of the frame on screen, arrow refers to the algorithm of the arrow on screen. Select arrow when the current is only line-tracking algorithm, for others, choose frame.

⑦ Get the parameter of the IDx from the “result”. If this ID is not in screen or unlearned, return -1.

Step3: Flowchart Analysis

3. Sample Program

4. Result

The camera learns a face in advance. After uploading the above program to the main control board, when the camera sees the specified face, it will show a smiling face, and when it sees other faces, it will switch to a crying face.

* When running the program, in addition to powering the main control board, the camera also needs an independent power supply.

Task 2: Adding Music

1. Program Design

Here we can add two pieces of music, one is beautiful while the other is funny.

(1) How can we add music? Here we can use the buzzer in the kit. When using, just turn on the buzzer switch.

(2) How can we find the music score? Search on the internet and select a piece of music you like.

(3) How do we program? In Mind+, the instruction of playing notes is built-in, and there are low, medium, high pitches, and various beats.

(4) How can we correspond a score to a command? As shown in the table below, here is a simple way to identify, taking note 2 for example.

2. The Sample Program

3. Operating Effect

Let the camera learn your face in advance, and upload the above program to the main control board. When detecting your face, it will play the beautiful music. If the camera catches other faces, it will play the funny one. Isn’t it really funny?

Finally, you can find a small mirror, hide the hardware behind the mirror, expose the camera to collect facial information, and build a magic music mirror.

Project Summary:

Through the music mirror project, we learned the working principle of face recognition and the application of instruction of face recognition algorithm on HUSKYLENS.

In AI visual recognition field, face recognition is an essential part of plenty of application scenarios. Let's think about what other face recognition applications we can make.

Review:

1. Learn the working principle of face recognition.

2. Learn the related instructions about face recognition of HUSKYLENS.

Project Development:

In this project, we designed an interesting music mirror by combining face recognition with music and some sensors.

When the camera recognizes a face that you don’t recognize, it emits a specified sound. Isn’t this an access control alarm system? However, if applied to the actual scene, the sound of the buzzer alarm is too low, so can we combine it with the Internet of Things? We can try building a home alarm system using the camera to detect whether there is a stranger outside the door. If the stranger stays too long and does not ring the doorbell to visit, the send messages to the master's computer or phone via IoT.

Next Class: Color Piano | Huskylens Playground with micro:bit EP02 >