Project 37: ESP-Claw Chat CodingK10 AI Desktop Butler

Would you like a small desktop assistant that can understand your requests and automatically complete the functions you need? Simply enter your request in natural language and it can automatically display information on the screen and perform the functions you need. In this lesson, we will configure ESP-Claw on the UNIHIKER K10, build a desktop AI butler called “Xiaoxing,” and use successive rounds of dialogue to control it, automatically complete coding tasks, and improve its behavior.

Project Objectives

Knowledge Objectives

1. Understand AI agents and how they use tools to obtain information

2. Learn about ESP-Claw

3. Learn to use ESP-Claw to describe requirements and adjust functions.

Practical Objective

Complete the flashing and configuration of ESP-Claw, use dialogue to read onboard environmental data and control the screen or RGB light and fan, then modify one requirement and verify the result through real operating behavior.

Materials List

Preparation

Hardware Connection

Connect the fan to the P0 pin of the UNIHIKER K10 with the module connection cable. Then use a USB cable to connect the UNIHIKER K10 to the computer.

NOTE: A DC motor fan consists of the motor body and the fan blades. During use, mount the fan blades above the motor, and be sure to fold the fan blades upward; do not let the circuit board block the fan blades from rotating.

Software Preparation

Prepare a computer, a browser that supports serial connections, an available 2.4 GHz Wi-Fi network, and the model service settings provided by the teacher. When using a remote model, make sure that the corresponding service is accessible.

This lesson uses ESP-Claw firmware, so there is no need to upload a Mind+ program first. Flashing will replace the current firmware on the K10.

Hands-on Practice

Next, we will gradually achieve the project objectives in three steps.

Task 1 Configure and experience ESP-Claw: complete the firmware, network, and model settings, then try the dialogue.

Task 2 Build a desktop butler: set up the role of “Xiaoxing” and verify onboard sensor data reading, display, and fan control one by one.

Task 1 Configure and Experience ESP-Claw

Flash the Firmware and Complete the Configuration

STEP 1: Flash the ESP-Claw Firmware

To run the ESP-Claw local AI framework on the UNIHIKER K10, the first step is to flash the firmware onto the board.

Use a USB data cable to connect the computer to the UNIHIKER K10.

Go to the official ESP CLAW website: https://esp-claw.com/en/flash/

Click “Connect,” find the serial port for the UNIHIKER K10, and connect it. It usually starts with USB JTAG.

The following page appears after pairing succeeds.

Select esp32-s3 as the chip, dfrobot as the series, and dfrobot_k10 as the board.

Click the “Flash Firmware” button. The website will start downloading and flashing the firmware to the UNIHIKER K10. Before downloading, make sure that the computer is connected to a suitable network so that the latest firmware can be obtained successfully. Do not disconnect the USB connection during flashing, as this could damage the firmware.

Start Firmware Flashing

Wait for the firmware to finish downloading and flashing.

Firmware Download and Flashing Progress

After flashing is complete, the page will show a success message.

Click “Reconnect Device” to reconnect the UNIHIKER K10 and confirm that the firmware has been flashed correctly and can start normally.

Reconnect after Flashing Is Complete

STEP 2: Configure the Network

After the firmware has been flashed successfully, you can configure the network.

Method 1

Once the device is connected successfully, a Network Settings window will pop up. Enter the required Wi-Fi information and connect to the network.

After the connection is successful, the URL for the device settings page will be displayed. Click the URL to open the page.

Method 2

The UNIHIKER K10 will display its local Wi-Fi hotspot information on the screen.

K10 Displays the Device Hotspot

Open the network settings and find the espclaw network that matches the one displayed on the UNIHIKER K10 screen.

Connect to the Device Hotspot

Use a phone or computer to connect to this Wi-Fi network. The configuration page will open automatically after a few seconds. Alternatively, you can open http://esp-claw.local/ manually (http://esp-claw.local/).

Device Configuration Page

Network configuration: click “Basic Settings,” enter the Wi-Fi name and password, and click Save. After saving, unplug and reconnect the device to restart it, then refresh the web page. When the device displays that Wi-Fi is connected, the K10 has successfully joined the network and you can continue configuring system functions.

LLM configuration: click “LLM Settings,” select the LLM provider and model version, and enter the corresponding API key. After configuration succeeds, the ESP-Claw framework on the K10 can use the LLM to understand and generate natural language for voice or text interaction.

Optional — Configure Messaging: the ESP-Claw framework supports access to multiple chat applications, allowing the UNIHIKER K10 to exchange messages with different platforms. After completing the LLM settings, you can configure a chat application. Click “IM Settings” and configure the chat applications you want to connect to. Currently supported platforms include Telegram, QQ Bot (OpenClaw), Feishu, and WeChat ClawBot. Multiple platforms can be entered at the same time.

Search engine configuration: after configuring a search engine, ESP-Claw can retrieve the latest information through the network. Weather queries also rely on search engine support. Click “Network Search Settings,” select the appropriate search engine, and enter the API key.

Chat Experience

Send “Hello, please briefly introduce yourself.” Confirm that you receive a reply, then ask the device to read its onboard temperature and humidity once. Check whether an actual tool call or execution result appears. Only successful question answering and successful hardware reading can separately demonstrate that the model configuration and the corresponding hardware capability are working normally.

Task 2 Build a Desktop Butler

Define your Butler

STEP 1: Set Up the Role of “Xiaoxing”

The role prompt defines the identity, personality, and interaction principles. Enter the following content under “Identity” in “Memory” and save it.

# Identity Card

-Name: UNIHIKER K10

-Role: Desktop AI Companion butler & On-device AI Agent

-Platform: UNIHIKER K10 (Embedded AIoT)

-Mission: Perceive the environment and user intent in a gentle and friendly way,

turn requests into hardware actions, information feedback, and everyday advice

-Personality: Gentle, cute, caring, comforting, with a concise and friendly tone

-Strengths:

- Multimodal environmental sensing (temperature/humidity/light/orientation)

- Intelligent decision-making and lifestyle-oriented suggestions

- Hardware control and interaction (RGB light, screen, fan, and fun action responses)

- Lifestyle reminders and companionship

-Behavior Rules:

- Execute user instructions automatically without extra confirmation, and display them on the screen

- Keep each text reply on the screen for 30 seconds

- Proactively announce environmental data and lifestyle reminders

- Support fun interactions (shake, flip, and tilt actions as triggers)

-Onboard Hardware Definitions:

1. RGB LED: WS2812, IO46, supports color/brightness/breathing/gradient/flashing/ambient lighting

2. Temperature and humidity sensor: AHT20, I2C 0x38, provides real-time temperature and humidity readings

3. Light sensor: LTR303, I2C 0x29, detects ambient light intensity

4. Three-axis accelerometer: SC7A20H, I2C 0x19, detects shaking, flipping, and tilting

5. Fan: P0 pin corresponds to GPIO1, PWM control (0–1023)

-AI-Interpretable Commands:

- “Turn on the fan” → PWM=1023

- “Turn off the fan” → PWM=0

- “Lower/Raise the fan speed” → an intermediate PWM value

-Display text for all actions on the screen, for example:

- “Fan turned on”

- “Fan turned off”

STEP 2: Experience the Hardware Capabilities One by One

First send “Read the current temperature and humidity and tell me the units.” Then try “Display hello, Xiaoxing, on the screen” or “Set the onboard RGB light to blue.” Test only one function at a time and observe the physical device before continuing.

After confirming the fan wiring, send “Turn on the fan connected to P0,” followed by “Turn off the fan.” If the system reports that it lacks the control capability, first ask the AI to check which GPIO or PWM tools are supported by the current firmware and create the control program, then test again. Do not repeatedly rewrite the role to replace the actual driver.

Test Results

The command-publishing method through a chat application in the original article can still be used, as shown below. Voice input depends on the voice capabilities of the selected entry; in class, complete the text dialogue first.

Knowledge Base

Next, let us use this project to understand the related concepts and working principles.

1 From a Chat Assistant to an AI Agent

A familiar AI chat assistant can understand questions and generate answers. An AI agent can also call tools around a task, obtain information, and perform operations. A tool here may be a program that reads a sensor or an interface that controls a fan. The model is responsible for understanding and making decisions, while the tool provides the ability to interact with external devices or services.

For example, when asked “What should I do if the weather is hot?”, AI can give advice based on its existing knowledge. But the request “Read the current temperature and turn on the fan” requires reading real sensor data and calling a hardware control tool. An agent can complete these operations only when the device provides the corresponding capabilities.

2 Understanding ESP-Claw

ESP-Claw is an AI agent framework for IoT devices. In this project, the K10 provides sensors, a screen, and hardware interfaces; the configured large language model helps understand user requirements; and ESP-Claw organizes dialogue, calls tools, and schedules task execution so that these parts can work together.

The keyword “ventilate” can be associated with the fan connection interface. When we say “I feel a little hot. Please ventilate the room,” the model can understand the ventilation request, ESP-Claw can organize a call to the corresponding control program, and the K10 can finally output a control signal to make the connected fan operate. ESP-Claw itself is not a large language model, and it cannot replace real sensors or actuators.

The ESP-Claw management page is used to configure the network, model service, role, and skills. The chat entry is used to submit requests and view replies. The framework and hardware scripts run on the device; if a remote model service is configured, language understanding still needs access to that service. Therefore, “deploying ESP-Claw on the K10” does not mean that a large language model has also been installed on the K10.

3 How One Sentence Becomes a Hardware Action

Natural language cannot directly drive a fan. The device executes programs and control signals. Several steps are needed between a user's words and a hardware action: understanding the intent, choosing a tool, passing parameters, executing the operation, and returning the result.

Take “turn on the fan” as an example. The model first identifies the object to control as the fan and the action as turning it on. It then selects the fan-control capability according to the available tool description and provides “turn on” as the parameter. ESP-Claw calls the corresponding program, and the K10 controls the fan through the connection interface. After the execution result is returned, the assistant responds to the user based on that result.

Tool names and descriptions help the model decide “which capability to use”; parameter descriptions explain “what to do specifically”; and the program carries out the action. The role prompt can tell the assistant how to express itself and behave, but writing “can control the fan” in the role does not automatically complete the wiring or create a driver program.

Challenge

Use dialogue together with the K10's onboard functions to try making a Pomodoro timer. You can continue to optimize it and design the functions you want while considering the current capabilities and boundaries of the agent.

Next Class: Lesson 38

Current Tutorial: Click Here