Lesson 3 Colorful Piano with an AI Machine Vision Sensor
Introduction
Color recognition technology in the field of artificial intelligence has a wide range of applications in daily life and work, from artistic creation to machine control. In this tutorial, we will build a project called "Colorful Piano" by combining micro:bit with AI machine vision sensor HUSKYLENS. This project mainly utilizes the color recognition function of AI machine vision sensor HUSKYLENS to identify different colors and play corresponding notes according to the recognized color.
Learning Objectives
1. Learn the operating principles and application fields of color recognition.
2. Learn to use the color recognition function of HUSKYLENS.
3. Learn to make a color piano using HUSKYLENS.
Preparation
Learning Content
In today's society, automated production has become a social development trend. Machine vision, as the eyes of "robots", is particularly important.
As one of the important technical directions, color recognition has undergone multiple generations of technology upgrades. And this project is to use the color recognition function of the AI machine vision sensor HUSKYLENS sensor to distinguish and recognize colors, and play the piano according to different colors.
Working principles and applications of color recognition
1. What is color recognition?
What is color recognition? First of all, we must understand what color is.
Color is a visual effect on light produced by our eyes, brain, and our life experience. The light we see with the naked eye is caused by a very narrow wavelength range produced by electromagnetic waves. The electromagnetic waves of different wavelengths appear in different colors. Color recognition is based on the color attributes under different brightness to recognize and distinguish.
2. The principle of color recognition
Color recognition is based on Lab color space, with dimension L representing brightness, a and b representing opposite dimensions of color, which is based on the CIE XYZ color space coordinates of nonlinear compression. We can regard these three parameters of Lab as the XYZ of the three-dimensional coordinate system. Compare the Lab parameters of the recognized and learned colors. When the two colors match within a certain error range, they are judged to be the same color.
In our usual use of color recognition, the hue and saturation in the color attributes of the same module are fixed, but the brightness will change due to the influence of the ambient brightness, so you must try to ensure that when using the AI machine vision sensor HUSKYLENS color recognition function, the environment brightness during learning and recognition is consistent with that during actual work.
3. Application Scenario of Color Recognition
1) Color recognition is now widely used in industries such as printing, coatings, and textiles for color monitoring and calibration.
2) Help people with color weakness or visual impairment to recognize and enhance their understanding of color.
AI Machine Vision Sensor HUSKYLENS Color Recognition Demonstration
If you want a color piano to play smoothly, first you should let the AI machine vision sensor HUSKYLENS sensor learn the color of the keys, and let it know the notes corresponding to each color. The color recognition function in the AI machine vision sensor HUSKYLENS sensor uses the sensor's built-in algorithm to identify the ID of different colors and feed them back to the main control board by learning and recording different colors.
The AI machine vision sensor HUSKYLENS sensor is set by default to learn, recognize and track only one color, but there is not only one key on the piano, so we need to set it to recognize multiple colors.
1. Operational Settings - Learn Multiple:
1. Dial the function button to the right or left until the word "Color Recognition" is displayed at the top of the screen.
2. Long press the function button to enter the parameter setting of the color recognition function.
3. Dial the function button until "Learn Multiple" is displayed, then short press the function button, and dial to the right to turn on the "Learn Multiple" switch, that is, the progress bar turns blue and the square icon on the progress bar moves to the right. Then short press the function button to confirm this parameter.
4. Dial the function button to the left until "Save & Return" shows. And the screen prompts "Do you want to save the parameters?" Select "Yes" in default, now short-press the function button to save the parameters and return automatically.
In this way, the multiple learning function is set up.
2. Learning and Recognition
1）Color detecting
Point the icon "+" in the center of the AI machine vision sensor HUSKYLENS screen to the target color block, and a white frame will appear on the screen, which selects the target color block automatically. Adjust the angle and distance of the AI machine vision sensor HUSKYLENS to the color block so that the white frame can include the entire target color block as far as possible.
2）Color Learning
When detecting colors, press the“learning button” to learn the first color, and release it to finish learning. Meanwhile, a message “Click again to continue! Click other buttons to finish” will be displayed. Please short press the "learning button" before the countdown ends if you want to learn other colors. If not, short press the "function button" before the countdown ends, or do not press any button to let the countdown ends. The color ID displayed is consistent with the order of learning, that is, ID will be displayed as “ID1”, “ID2”, “ID3”, and so on, and different colors correspond to different colors of frames.
3）Color recognition
When encountering the same or similar color blocks, a color frame with an ID will be automatically displayed on the screen, and the size of the blue frame is the same as the size of the color blocks. Multiple colors can be recognized and tracked at the same time and different colors correspond to different colors of frames.
When multiple blocks of the same color appear, the separated color blocks cannot be recognized at the same time, and only one block can be recognized at one time.
Note: Color recognition is greatly affected by ambient light. Sometimes AI machine vision sensor HUSKYLENS may misidentify similar colors. Please try to keep the ambient light unchanged. In the process of using color paper card, you can put a piece of white paper under the paper card to reduce the influence of ambient light on color recognition.
AI Vision Project Practice
We will complete the task with two steps. First, we will learn to use the color recognition function of AI machine vision sensor HUSKYLENS and the output of the recognized color ID. Then we can play their corresponding sound according to the output color ID, so that we can complete our color piano.
Task 1: Multiple Color Learning
At the very beginning, we need to allow the AI machine vision sensor HUSKYLENS camera to recognize multiple colors, distinguish the differences in these colors, and be able to give feedback so that we can add notes later.
Task 2: Define Notes for Each Color
After being able to accurately identify each color, we can define a sound for each color, so that they can be played according to a certain rule, so that a color piano can be realized.
Task1: Multiple Color Recognition
Hardware Connection
AI Vision Program Design
Function Instruction:
Here we need to let the AI machine vision sensor HUSKYLENS sensor learn the color of each key and be able to output the color ID so that we can play the corresponding sound with the corresponding color. Such as: when it recognizes red (ID1), play the note Do.
Note: You need to enable the learning function first.
Flowchart Analysis:
The MakeCode Sample Program
MakeCode Sample Program link: https://makecode.microbit.org/_c9VagqJTdHmY
AI Vision Project Operating Effect
When the ID of the corresponding color is recognized in the AI machine vision sensor HUSKYLENS, the corresponding number will be displayed on the micro:bit.
Task 2: Define Notes for Each Color
AI Vision Program Design
Function Instruction:
How to write a program? In Mind+, it comes with instructions for playing notes, which are divided into low, mid, and high notes, as well as various beats. We just need to add the corresponding note to the corresponding ID.
Flowchart Analysis:
The Makecode Sample Program
Add the note modules to the completed task 1 program.
MakeCode Sample Program link: https://makecode.microbit.org/_TrpL7LaKAcxm
AI Vision Project Operating Effect
Run the program and find a music score, AI machine vision sensor HUSKYLENS will play the corresponding notes according to the recognized colors.
Question 1: In the process, you must have discovered that when AI machine vision sensor HUSKYLENS detects multiple colors, the notes produced by the color piano may not the notes we want. Do you have a good way to solve this problem?
Question 2: In the process, the color recognition will consider the similar color as the same color. How should we solve this problem?
Tip: Adjust the threshold of the recognition frame. In the submenu parameter setting interface of the color recognition function, there is a "Recognition Frame Threshold" parameter. The lower the value of this parameter, the lower the accuracy, but the more similar color blocks are recognized. As shown in the figure below, when the threshold is 20, only one yellow ball can be recognized; when the threshold is 0, all three yellow balls can be recognized. The threshold of the recognition frame can be adjusted according to the actual effect, so that the recognition accuracy is within an acceptable range.
Project Review
In this lesson, we learned about the working principle of color recognition, learned how to make a color piano, and learned the color recognition function.
Color recognition is a very important function in artificial intelligence visual recognition, and it has been widely used in the industry. Think about what other interesting functions can be achieved by color recognition?
Project Development
After completing the color piano, we will find a problem that the number of keys is relatively small. If we want to increase the number of keys, as the color increases, there will be many keys with similar colors, which may cause misidentification. If there are more keys, the camera's recognition range will not cover them all. So, is there any way we can broaden the range of our color piano?
(tips: We can use the A and B button on the micro:bit to realize the upgrade and downgrade functions.)
Click Buy Study Pack of HUSKYLENS for micro:bit
Click Buy Huskylens - An Easy-to-use AI Camera | Vision Sensor
Click Buy Study Pack with HUSKYLENS and micro:bit V2
Engage in learning the next AI vision project：Vending Machine for Stray Cats & Dogs
Engage in learning the previous AI vision project : Face Recognition Access Control with an AI Machine Vision Sensor
