Lesson 11: AI Traffic Surveillance Eye

The "electronic police" at zebra crossings has been in service for quite some time, mainly to capture motor vehicles that fail to yield to pedestrians. Of course, most drivers say they are willing to yield proactively, but some also raise concerns: sometimes pedestrians cross carelessly, make phone calls, look at their phones, or run red lights. In such situations, how can we ensure that both vehicles and pedestrians pass in an orderly way?

Next, we will use a UNIHIKER M10, a PIR motion sensor, and a USB camera to build an AI pedestrian traffic e-eye. Acting as an "electronic police officer" supervising zebra crossing behavior, it monitors pedestrians and helps promote safe and civilized travel.

Goal

On the UNIHIKER M10, the display mode switches every 30 seconds. When the light is red, if the PIR sensor at the zebra crossing detects a pedestrian, the system captures that person and displays the photo on the UNIHIKER M10.

Get to Know

1. Master how to take photos with a USB camera.

2. Understand how a USB camera works.

3. Master how to view system files on the UNIHIKER M10.

Materials List

Software: Mind+ programming software x1
Download: https://mindplus.cc/en

Hands-on Practice

The AI pedestrian traffic e-eye is used to capture pedestrians who violate crossing rules during a red light, take their photos, and display them on the UNIHIKER M10. Let us

implement it through the following subtasks.

Task 1: Human-Sensing E-eye

When the PIR motion sensor detects a person, use the USB camera to take a photo and display it on the UNIHIKER M10.

Task 2: Design Traffic Lights

On the UNIHIKER M10, switch among red, yellow, and green lights based on a countdown.

Task 3: AI Pedestrian Traffic E-eye

When the light is red, if the PIR motion sensor detects a pedestrian on the zebra crossing, the camera takes a photo and displays it on the UNIHIKER M10.

Task 1: Human-Sensing E-eye

1. Hardware Connection

A USB camera is a driver-free camera that works once plugged into a USB port. Connect the camera to the USB port of the UNIHIKER M10, connect the PIR motion sensor to pin P24, and after hardware connection is complete, connect the UNIHIKER M10 to the computer with a USB cable.

Note: For details about the "USB camera," see the Knowledge Base.

2. Software Preparation

Open Mind+（1.8） and complete software preparation as shown below.

Add the OpenCV library: click "Extensions," find "OpenCV" in the official library, and add it.

3. Write a Program

To implement camera capture, first use the block to create a VideoCapture object named `vd`.

After the object is created successfully, use the block "VideoCapture object `vd` open video/device" to open the camera.

Before taking photos, use the block to check whether the VideoCapture object has been initialized.

After the camera opens successfully, determine whether the PIR motion sensor detects a person (P24 = 1). If a person is detected, use the block "grab next frame from VideoCapture object" to capture one frame, then use the block "save image" to store it as `Mind+.png` in the UNIHIKER M10 system files.

Note: For how to view images saved on the UNIHIKER M10, see the Knowledge Section.

After a photo is captured successfully, you can display it on the UNIHIKER M10. First, import any image from the image asset library into the file system.

Use the block "display image" together with the block "update object numeric parameter width" to display the imported `.png` image on the UNIHIKER M10.

The UNIHIKER M10 is now showing the imported image. How can you show the image captured by the camera instead? Use the block "update image" and set the image source to `Mind+.png`.

4. Run the Program

Click Run. After successful execution, trigger the PIR motion sensor, and the UNIHIKER M10 will display the camera-captured image in real time.

Task 2: Design Traffic Lights

Before programming, let us analyze what functions are needed in this task. First, design a simple interface. Then complete the display of red, yellow, and green lights. Finally, switch traffic lights according to a countdown.

1. Write a Program

（1）UI Interface Design

To simplify the program, the frame and fixed text in the interface are made into a background image. Import this background image from the image asset library into the file system.

At the start of the Python main program, use the block "display image" to show the background image on the UNIHIKER M10.

Use the block "fill circle": first fill the red light circle in red, then fill the yellow and green light circles in white.

On the right side of the traffic light, add a countdown display by using the block "display digital tube font," set content to 30 and color to red.

How can we switch traffic lights? Create three functions respectively: red_light, green_light, and yellow_light.

In the red-light function, use the block "update color" to set object `hong` to red and set the other two objects to white. Use the same method in the other two functions.

To switch signal lights of different colors, call the red-light and green-light functions in a loop block. Using 30 seconds as an example: call the red-light function, wait 30 seconds, then call the green-light function and wait 30 seconds. This achieves red for 30s then green; green for 30s then red.

In real life, traffic waiting time is often shown as a countdown. Next, replace the "wait 30 seconds" block in the program with a countdown display. Use a `for` loop block, set variable `my variable` from 30 to 0 with step 1, then use the block "update text content parameter" to update the countdown to `my variable`.

When 5 seconds remain in the countdown, switch from red to yellow. Use a conditional block to check whether `my variable < 6`; if true, call the yellow-light function.

Now use the same method to implement the green-light countdown display. The complete program is as follows:

2. Run the Program

Click Run. After successful execution, the UNIHIKER M10 displays traffic lights, and they switch red-yellow-green-yellow based on the countdown.

Task 3: AI Pedestrian Traffic E-eye

Task 1 implemented a human-sensing e-eye, and Task 2 implemented traffic light switching based on countdown. The AI pedestrian traffic e-eye is essentially the combination of Task 1 and Task 2, by implementing Task 1 functionality within the call to the red-light function in Task 2.

1. Write a Program

This task modifies the Task 2 program. At the start of the Python main program, use the blocks to create a VideoCapture object, open video/device with `vd`, and check whether the

VideoCapture object is initialized, to ensure the camera is ready.

When the light is red and the PIR motion sensor detects a pedestrian, the camera takes a

photo and displays it on the UNIHIKER M10. This means the camera capture logic should be

implemented where the red-light function is called. But how can the red-light switching

program and camera capture program run at the same time? Use the block "start thread" to

launch a thread under the red-light function call.

Under the block "when thread object starts," use a conditional block to determine

whether the PIR motion sensor detects a person (P24 = 1?). If a person is detected, use the

block "grab next frame from VideoCapture object" to capture a frame, and use the block

"save image" to store it in local files on the UNIHIKER M10.

To display the captured violation photo on the UNIHIKER M10, at the start of the Python

main program use the block "display image" and the block "update numeric parameter" to set

image position. In the thread, use the block "update image" to set the displayed image to

`Mind+.png`. The complete program is as follows:

2. Run the Program

Click Run. After successful execution, the UNIHIKER M10 interface switches traffic light display based on countdown. When the PIR motion sensor detects a person, the UNIHIKER

M10 display shows the camera image.

3. Have a try

There is still one small issue in the program above: during green light, if the PIR motion

sensor detects a person, the camera also captures and displays the photo on the UNIHIKER

M10. Next, modify the program so that camera capture and display are enabled only when

the light is red.

Hint: Under the program section where the green-light function is called, use the block

"thread object stopped" .

Knowledge Base

1. Understanding USB Cameras

In this lesson, we use a driver-free USB camera. As long as the camera is plugged into a

USB port, it can work.

How does a camera work? Here is a brief introduction. A scene passes through the lens

to form an image and is projected onto the internal image sensor of the camera. After

processing, the signal becomes an image file, which is then transferred to the UNIHIKER M10

through the USB interface. Finally, we can display the image on the UNIHIKER M10 screen or

download the image file to a computer for viewing.

2. How to View Images Saved on the UNIHIKER M10

In Task 1 and Task 3, we used the "save image" block. Where are these images saved? After

connecting to the remote terminal of UNIHIKER M10 at `10.1.2.3`, you can see a folder named

"Files in UNIHIKER M10" in the file system directory.

Click the `mindplus` folder, then click the `cache` folder, and locate where `Task1.mp` is saved. Click `Task1.mp`, and you will see two image files: `bg.png` and `Mind+.png`.

Among them, `bg.png` is the image file we imported manually (a project file), while `Mind+.png` is the image captured from the camera using the "save image" block. Right-click

`Mind+.png` and choose "Download file to computer" to save it locally. Remember the save

path so you can view the camera photo on your computer.

3. How to Access UNIHIKER M10 System Files from a Computer

1）. Enable File Sharing on UNIHIKER M10

Press and hold the "Home" key on the UNIHIKER M10 to enter the menu page, click "3 -

App Switch," and check whether file sharing is enabled. If it shows "Disabled," click the file

sharing option and switch it to "Enabled."

2）. Configure the Computer

On the desktop, open "This PC" by double-clicking. Enter the board's network path in the address bar in the format `\\IP address\`. When connected via USB, the IP address is `10.1.2.3`. Enter `\\10.1.2.3\` and press Enter to open the UNIHIKER M10 system folder.

In the UNIHIKER M10 system folder, there are `media` and `root` folders. `root` is the onboard storage of the UNIHIKER M10, while `media` is the external storage space (after inserting an SD card or connecting a hard drive via USB, it will be automatically mounted under this folder).

3）. View Images Saved on the UNIHIKER M10

Double-click the `root` folder. You will be prompted to enter an account and password.

Enter the corresponding credentials and click OK to access the `root` folder of the UNIHIKER

M10.

Account: root

Password: dfrobot

In the folder list, find the `mindplus` folder, then click the `cache` folder to locate where

`Task1.mp` is saved. Click `Task1.mp`, and you can likewise see two image files: `bg.png` and

`Mind+.png`.

4. Block Overview

Challenge Yourself

Observe traffic lights in daily life and you will find: during red light, the countdown color is

red; during yellow light, the countdown color is yellow; during green light, the countdown

color is green. Next, modify the program so the countdown color stays consistent with the

traffic light color.

Hint: Use the block "update color" and update the display color of the countdown object

in the corresponding function.