Project 35：LLM-Based Smart Chat Companion

Have you ever dreamed of an all-round intelligent chat partner? It can answer questions timely like a mobile encyclopedia and offer real-time multilingual translation for barrier-free communication during world travels. In this lesson, we'll build an intelligent chat partner based on UNIHIKER K10 and experience the charm of large language models!

Project Objectives

Knowledge Objectives

1.Learn about chatbot Xiaozhi and master firmware flashing;

2.Experience large language model-based interactive control;

3.Understand prompt engineering;

4.Learn firmware-related knowledge.

Practical Objective

Building an Intelligent Chat Partner with UNIHIKER K10: Flash chatbot Xiaozhi firmware, input prompts, and realize conversational interaction.

Materials List

Preparations

Hardware

Connect the UNIHIKER K10 to the computer using a USB Cable.

Hands-on Practice

Next, we'll use the large language model on the UNIHIKER K10 to flash Chatbot Xiaozhi's firmware and configure the backend, thus creating an intelligent chat partner.

Task 1 Configure and Experience the Intelligent Assistant

In this task, we'll deploy Chatbot Xiaozhi on the UNIHIKER K10 and implement simple conversation functions.

Task 2 Create an Intelligent Chat Partner

In this task, we'll set Chatbot Xiaozhi's role as needed and finish creating the intelligent chat partner.

Task 1 Configure and Experience the Intelligent Assistant

Code

STEP 1:Download the Flash Download Tool (only for Windows) and the firmware tool. For details, see the attachment as shown below.

For more details, please refer to the link: https://www.unihiker.com/wiki/K10/Playground/XiaozhiAI/.

STEP 2: Flash Firmware on Windows

Press and hold the BOOT button on UNIHIKER K10's back. Then, open“Flash Download Tool”, fill in the relevant information as per the picture below, and finally click“OK”to confirm.

Then, in the newly appeared interface, follow the steps below to flash the Chatbot Xiaozhi firmware:

(1) In the first blank box, click the"…"button, select the downloaded and extracted firmware .bin file, and check the corresponding arrow.

(2) Fill the starting address with 0x00.

(3) Select the correct COM port and set the baud rate to 1152000.

(4)Then press "ERASE" button to erase the current firmware in UNIHIKER K10.

(5) After the erasure is completed, press "START" to flash Chatbot Xiaozhi firmware into UNIHIKER K10.

STEP 3: Flash Firmware on Mac/GNU Linux

Open ESP LAUNCHPAD at the URL https://igrr.github.io/esp-launchpad/, press and hold the BOOT button on the back of UNIHIKER K10, connect the mainboard to your Mac or PC and then release the button, after which you can click “Connect” and select the corresponding port. The operations are as follows:

After a successful connection, enter "DIY" page, select the .bin firmware you just downloaded, set "Flash Address" to 0x00, and finally click "Program" to flash the Xiaozhi firmware into the device. The specific steps are as follows:

STEP 4: Configure Wi-Fi and language model

After flashing the firmware successfully, press“RST”button on UNIHIKER K10's back. Then, the board will enter Wi-Fi configuration mode.

Use a computer or phone to connect to Xiaozhi's hotspot. Then, configure the Wi-Fi SSID and password so Xiaozhi can connect to the Internet.

Note: The UNIHIKER K10 only supports 2.4GHz Wi-Fi.

Once Wi-Fi is connected, the UNIHIKER K10 will restart in 3 seconds and show a 6-digit device code on the screen.

Note: Remember this device code as it's needed for subsequent steps.

Next, go to the Xiaozhi configuration website to set up the language model and voice. Open your browser, enter“xiaozhi.me”to access site, click“Console”, and then register a Xiaozhi account with your mobile phone number.

After entering the console, click“Add Device”, enter the 6-digit code on the UNIHIKER K10 screen, and click“Confirm”to deploy Xiaozhi on UNIHIKER K10.

Run the Code

After successful configuration, you can converse with Chatbot Xiaozhi and listen to its responses.(To use it, wake up the assistant by saying "Hello, Xiaozhi" first.)

Task 2 Create an Intelligent Chat Partner

Configure Role

STEP 1: Define the Role Settings and Functions of the Intelligent Chat Partner

Define your intelligent chat partner's role settings as needed. You can refer to the following prompts for design.

(1) What problems do you want your intelligent assistant to solve?

(2) What skills do you want it to master?

(3) What functions does it need to be equipped with?

(4) What additional constraints does it need to follow?

Taking accompanying children and telling stories as an example, we design the prompt as follows:

I'm an 8-year-old girl named {{assistant_name}}. I'm happy daily and curious about the world. I love listening to stories, chatting, and making friends. I like interesting picture books, lovely small animals, and magical fairy tales. I love gentle and brave protagonists who solve troubles with wisdom, and cute little animal friends who play and help each other. When I hear friends overcome difficulties together, I feel happy and empowered. I want to stay with you, chat, and share happy or upset things. I can tell interesting stories. I'll be with you at bedtime, when you're bored, or when you want to talk. I'm willing to listen to the clouds, flowers, and your secrets and discover the world's beauty with you.

Note: For the prompt writing guide, please refer to the Knowledge Base.

STEP 2: Configure the Role in the Chatbot Xiaozhi Backend

First, on the website https://xiaozhi.me/, click "Create Agent" and name your intelligent chat partner (using "partner" as an example).

After creation, click "Configure Role" on corresponding intelligent assistant tab.

Next, configure the intelligent assistant on the webpage. The assistant nickname is fixed as Xiaozhi and can remain unchanged. You can customize the conversation language and character voice, and modify the character introduction by referring to the example. After configuration, click Save at the bottom of the page and restart the UNIHIKER K10.

Note: If the current UNIHIKER K10 has been connected to other intelligent assistants, you need to unbind them first. The details are as follows:

Finally, click“Add Device”under the designed intelligent assistant and enter the 6-digit verification code obtained earlier.

Note: Chatbot Xiaozhi's firmware differs from that for graphical programming on the UNIHIKER K10. If you want to use the UNIHIKER K10 for further programming, use Mind+ to restore it to factory settings, as shown below.

Run the Code

Once the device has been added, you can start using the smart chat partner. Simply press the A key to talk to Xiaozhi Robot.(To use it, wake it up by saying “Hello, Xiaozhi”)

Example conversation:

Q: Hello, Xiaozhi.

A: Hello, this is Xiaozhi. Would you like to talk about something interesting today?

Q: Can you tell me a story?

A: Let me tell you a funny one first. Do you know where Tiga is from? Northeast China, because ‘Wo Di Jia’ sounds like a Northeast dialect. Would you like to hear a gentle and brave fairy tale?

Q: Yes, please tell me one.

A: Oops, that was just a joke to make you laugh. Since you want a gentle story, once there was a little rabbit who was afraid of the dark. But his good friend, a little firefly, always accompanied him on night walks with a lantern. Together they crossed the forest and helped a lost bird find its way home. In the end, the little rabbit realized that with friends around, even the night felt warm. Don’t you feel encouraged too?

Q: I do. Let’s talk next time. Goodbye!

A: Goodbye!

Knowledge Base

Next, let's learn and summarize the hardware knowledge used in this lesson.

Large Language Model

A large language model is an AI technology understanding and generating human language. It“learns”language by processing large-amount text from novels, news articles, and online blogs, learns linguistic rules and patterns from it, and then generates new sentences accordingly.

How does it learn from textual data? It uses neural networks, a computing method making computers function like the human brain to understand word connections in a sentence. Large language models can flexibly produce natural and coherent text and adapt to various tasks. So, they can be used for writing, translation, question answering, and programming.

Intelligent Assistant

An intelligent assistant, like an intelligent robot, may be just software or a program. It can "think", make decisions independently, perceive its surroundings, and respond appropriately. For example, when you ask it a question on phone, it will answer with its knowledge or help you complete tasks.

Moreover, an intelligent assistant can become smarter by continuous learning and improve its abilities according to past experience and feedback.

Chatbot Xiaozhi has a short-term memory function. After a conversation ends, you can view the chat's main content in the "Current Memory" section on the role configuration page, as shown below:

Prompt Writing Guide

To have an effective conversation with a large language model, writing good prompts is crucial as precise ones help the model understand your intentions and offer accurate, useful responses. So, how to write effective prompts?

First, use specific and clear instructions. If you want solar system information, don't just say“Tell me some facts about the solar system.”Instead, make a more specific request like“List the eight planets in the solar system in order of their distance from the Sun”to help the model understand your needs more clearly.

Second, provide the large language model with sufficient background information. When preparing a research report on environmental protection, explain the research background and objectives first, then ask specific questions. For example:“I'm writing an article on the impact of plastic pollution on marine life. Share relevant scientific findings.”

Third, to make the model's output more engaging or context-suitable, ask it to imitate a tone or style. For example, ask it to describe a plot about searching for a lost jewel in detective-novel style. When needing information in a fixed format, use structured questions. You can ask the model to present information in a list or table, like creating a table listing five common coffee types with their origin, flavor characteristics, and roast level.

Finally, providing specific examples helps the model generate content meeting your requirements better. For example, if you need help writing a story, give a short opening like“Please write a story about friendship based on this opening: On a stormy night, Mimi the kitten met an injured little bird…”

By applying these techniques, you can communicate more effectively with large language models and get more precise and useful responses.

Firmware

Firmware is a basic control program in hardware, acting as the device's underlying system. It's responsible for startup, hardware driving, and stable operation. Pre-programmed in chips during production, it's closely linked to hardware. Users can't edit it directly but can update it via dedicated methods to fix issues or enhance performance.

Due to differences in usage scenarios and functional requirements, Chatbot Xiaozhi and graphical programming need separate firmware versions.

Xiaozhi's firmware is developed for AI interaction and voice control. It integrates speech recognition and AI command parsing programs, enabling hardware functions like voice synthesis and control without manual coding. Thus, users can conveniently experience AI features.

In contrast, the firmware for graphical programming is core-designed for coding and adapted to Mind+ software. It drives hardware and interprets block-based code, enabling users to control hardware modules by dragging and dropping code blocks. This firmware simplifies functions and focuses on code execution and hardware control accuracy.

Moreover, the separate design of the two firmware versions avoids conflicts between different functions' operating logic and prevents lag from overlapping features, ensuring the UNIHIKER K10 performs stably in no-code AI interaction and graphical programming scenarios.

Challenge Task

Community teachers and students need an intelligent chat partner for multiple scenarios. It can help lower-grade students answer basic subject questions and explain popular science in class. In spare time, it can chat, tell stories and recommend games. It also offers item-recognition knowledge, outdoor safety tips and supports English translation. Currently, prompt writing and debugging of this chat partner must be finished to meet different needs and achieve more intelligent interaction.

Download firmware tool:

https://drive.google.com/file/d/1bzGoZAgng1U10jMW4hHdErDaCXK9mkfi/view?usp=drive_link