Can a robot vacuum navigate around obstacles? 

Can it accurately recognize steps of varying heights?

 Can it find its charging dock in a cluttered environment? 

These questions, once answered largely through individual company testing, are now being guided by increasingly unified evaluation standards.

Part 01
International Standards Take Shape: From What to Test to How to Test

Recently, the international standard "Performance evaluation methods of robots for household and similar use" (IEC 62849), developed under China's leadership, was officially published. Designed to address the real-world usage needs of household robots, the standard defines 11 core test items, including obstacle avoidance, ramp operation, and energy consumption, providing a more unified basis for robot performance evaluation.

As the "what" to test becomes standardized, another question emerges: How do we know how well a robot actually performs?

The answer might be found right at the testing site.

In a robot testing facility featured on CCTV, a robot vacuum is undergoing performance tests across various scenarios – navigating through obstacles simulating table and chair legs, identifying steps and platforms of different heights, locating its charging dock in a complex environment, and completing motion coverage tests within a designated area. Around the testing field, an array of CHINGMU motion capture cameras is tracking reflective markers attached to the robot vacuum in real time with sub-millimeter precision.

From the minimum distance between the robot and obstacles in obstacle avoidance tests, to real-time changes in body posture during ramp tests, and deviations between actual trajectories and planned paths in motion coverage tests – across the 11 core test items defined by the international standard, CHINGMU's optical motion capture system provides high-precision external measurement data, offering an objective basis for evaluating robot performance metrics.

Part 02
Why Robot Evaluation Needs External Ground Truth

Robots already come equipped with numerous onboard sensors – cameras, encoders, IMUs, and more. So why is another measurement system needed during testing?

The answer lies in two words: Ground Truth.

Data from a robot's own sensors is, by nature, the robot's perception and estimation of its own state and the external environment. But to determine whether those perceptions are accurate, and whether control actions truly achieved the intended outcome, an independent reference benchmark outside the system under test is required. For example:

·The robot believes it moved forward 1 meter – how far did it actually move?

·The system planned a trajectory – how much did the actual execution deviate?

·The algorithm determined it completed area coverage – what was the actual coverage?

·The robot repeats the same task – is each execution consistent?

Through high-precision optical motion capture, the true position, posture, and motion trajectory of the robot and target objects can be continuously acquired within a unified spatial coordinate system, with further computation of velocity, acceleration, and other motion parameters. This data constitutes the critical external ground truth in robot performance evaluation.

The robot tells us "how well I think I performed"; external measurement tells us "how well it actually performed."

This is the foundation of objective evaluation.

Part 03
From Robot Vacuums to Humanoids: The More Complex, the More an Objective Ruler Is Needed

As the subject of evaluation shifts from household mobile robots to humanoid robots, the challenges grow significantly more complex. Today's humanoid robots are already running, dancing, playing ball games, performing complex manipulations, and entering an increasing number of competitive and real-world task scenarios. Whether a robot "can perform an action" is gradually becoming a basic question.

R&D teams increasingly need to know: Is the movement accurate? Is the motion stable? Is execution consistent across multiple repetitions? Does the end-effector trajectory meet the target? Where does the actual motion deviate from the standard? And how much performance gap emerges when simulation results are deployed to the physical robot?

In high-dynamic scenarios such as robot games, table tennis, and tennis, CHINGMU has been continuously involved in capturing and measuring robotic athletic capabilities. For humanoid robots in high-speed motion – joint angles, center of mass transfer, end-effector velocity at the moment of impact – all can be precisely reconstructed through optical motion capture.

In the past, when we watched robot competitions, the easiest things to see were: Did it win? Did it complete the task?

But from the perspective of robot R&D and testing, we also need to answer: Why did it complete it? What went well? What still needs improvement? How can it do better next time?

This requires translating the "movement performance" seen by the naked eye into measurable and analyzable motion data.

Part 04

From "Watching Performance" to "Measuring with Data"

In the past, when we watched robot competitions, the easiest things to see were: Did it win? Did it complete the task?

But from the perspective of robot R&D and testing, we also need to answer: Why did it complete it? What went well? What still needs improvement? How can it do better next time?

This requires translating the "movement performance" seen by the naked eye into measurable and analyzable motion data.

From R&D optimization to mass production delivery, and then to repair return and maintenance verification – this is a complete lifecycle for a robot. Along this lifecycle, evaluation is not a one‑time event but a continuous necessity. Every algorithm iteration, every hardware adjustment, every batch production validation requires objective measurement to answer the same question: "How well did the robot actually move?"

Leveraging high‑precision optical motion capture and robot motion analysis capabilities, CHINGMU has further launched the RoboEval Robot Motion Evaluation Platform. The core problem it aims to solve is straightforward: to turn "how well the robot moves" into a quantifiable question. By capturing the robot's actual execution through external high‑precision motion data, and analyzing it across dimensions such as trajectory deviation, motion differences, stability, and consistency, the platform enables R&D teams to gain a more objective understanding of the robot's true motion performance.

From real-world motion capture, to external ground-truth measurement, to motion metric analysis, and ultimately to robot capability evaluation – RoboEval is transforming robot motion performance from "looks good" to measurable, quantifiable, comparable, and verifiable.

But RoboEval is not an isolated evaluation tool. It is part of CHINGMU's broader RoboDecode Full Lifecycle Robot Platform architecture. Built on optical motion capture and multimodal perception, the platform comprises seven core modules: Capture, Integrate, Acquisition, Map, Train, Control, and Evaluate – forming a complete closed loop that spans real‑world data acquisition, training data processing, dataset construction, motion retargeting, Embodied AI training and learning, teleoperation control, and motion evaluation with quality validation. This end‑to‑end architecture supports continuous iteration across the entire robot lifecycle – from R&D optimization and mass production delivery to post‑repair retesting and maintenance verification. Within this closed loop, RoboEval is responsible for the "Evaluate" link: turning motion quality into data, and data into actionable insights for optimization.

From the establishment of international standards for evaluating the performance of household robots, to the increasingly complex motion capability testing of humanoid robots, a clear trend is emerging: as the robotics industry shifts from "can it do it" to "how well can it do it," the importance of evaluation continues to grow.

Standards tell the industry: what should be tested.

High-precision external measurement and evaluation tools need to answer the next question: how to test it accurately.

For CHINGMU, the value of motion capture is also expanding-from "recording how a robot moves" to objectively measuring a robot's true motion performance.

Because only when you can truly measure it, can you accurately identify problems, continuously optimize algorithms, and ultimately answer the increasingly critical question: "How well does the robot actually move?"

RoboEval-Contact Us Now

Scan the QR code to fill out the form, or call +86 13310099680. We'll get back to you shortly.