How smoothly can robots play? CHINGMU's mocap lights Hangzhou Robot Dream Night.

Source：

2025 Sep. 05

When robots play freely on piano keys, robot dogs perform lion dances to liven up the atmosphere, and drones depict the artistic conception of Tai Chi on stage... These scenes that once seemed only possible in sci-fi movies are now vividly presented at the "2025 Robot Dream Night". Co-hosted by the Zhejiang Robot Industry Development Association, Hangzhou Haichuang Humanoid Robot Industry Innovation Center, and Hangzhou Leji Robot Co., Ltd., this musical event offered the audience an audio-visual feast that blends a sense of technology with ornamental value. Among the highlights, the performance by the Hexuan Robot Band, featuring robots from Haichuang Humanoid Robot Industry Innovation Center, became a hit of the evening. The training process for their piano performances relied on high-quality motion datasets provided by CHINGMU's optical motion capture system, enabling the robots to learn and replicate human playing movements, thus demonstrating the unique value of motion capture technology in the field of humanoid robots. 

 Precise Replication of Fingertip Rhythm: Motion Capture Technology Makes Robots' Piano Playing More Authentic

 When it comes to robots playing the piano, many people have the impression of mechanical repetition and stiff movements. However, at this Robot Dream Night, the Hexuan Robot Band showcased by Haichuang Humanoid Robot Industry Innovation Center shattered this stereotype: they not only achieved precise collaborative performances across piano, yangqin (a traditional stringed instrument), and drums but also adjusted the strength of key presses according to the rhythm of the music, highly replicating the fluency of real human performances.

How did the robots achieve this breakthrough, moving from mechanical execution to human-like precise replication?

To help robots master the core skills of playing different instruments, CHINGMU provided customized optical motion capture solutions for robot training. Through high-precision optical motion capture equipment, the system captures detailed data of each finger joint, assisting the R&D team in building a rich database of human playing movements. After algorithm optimization and motion reproduction, these data ultimately enable the robots to break free from the limitations of mechanically executing commands, achieving flexibility and coordination in dual-arm collaboration. The precision of movements is refined to the fingertips, allowing the robots to perform complex multi-finger collaborative operations when playing different instruments, producing melodies that are both consistent with the characteristics of the instruments and possess both accuracy and fluency. 

CHINGMU's Optical Motion Capture System Remotely Operates Robots to Play the Piano

 As a platform for showcasing the technological achievements of Haichuang Humanoid Robot Industry Innovation Center nearly a year after its establishment, the "2025 Robot Dream Night" presented cutting-edge breakthroughs in the field of humanoid robots to the public. Currently, humanoid robots have become a focus of attention for national strategies, local governments, innovative enterprises, and universities. This concert is undoubtedly an excellent example of technology implementation, not only allowing the audience to intuitively witness the "evolution" of robots but also making them clearly feel that motion capture technology will become a key bridge between human wisdom and machine capabilities. 

Song Wei, Deputy Director of Haichuang Humanoid Robot Industry Innovation Center and Founder of Hangzhou Xuanji Robot Co., Ltd., stated: "The Hexuan Robot Band performing this time consists of three playing units—piano, yangqin, and drums—achieving collaborative performances across multiple instruments. The core technological highlights supporting their professional musical performances focus on three aspects: 

 First, high-reliability dexterous hands: simulating the fine movement capabilities of human fingers to ensure the accuracy of instrument playing; 

 Second, lightweight humanoid robotic arm collaborative control: achieving coordination and flexibility of dual-arm movements through algorithm optimization to replicate the fluency of real human performances;

 Third, intelligent music score analysis technology: relying on CHINGMU's motion capture system to collect human playing movement data, which is used as a training database for robots. After training with data-driven models, it automatically generates movement trajectories that match the characteristics of the instruments. 

CHINGMU's optical motion capture technology not only helps the team quickly and efficiently complete the debugging of robot motion control but also provides an important reference for the development of movements for humanoid dexterous hands and free robotic arms. Through the training and migration of real human movement data, a robot motion control library can be quickly built, which is expected to become the mainstream technical direction for humanoid robot motion generation in the future."

Song Wei, Deputy Director of Haichuang Humanoid Robot Industry Innovation Center and Founder of Hangzhou Xuanji Robot Co., Ltd., expounds on the significance of motion capture technology for humanoid robot research

 On the other hand, regarding the future technological exploration and commercialization path of intelligent robots, Director Song Wei pointed out: "With humanoid instrument playing as the core driver, the team will continue to deepen the hardware research and development of dexterous hands and humanoid robotic arms, as well as the optimization of motion control algorithms. The short-term goal is to achieve commercial implementation in artistic performance scenarios, forming a new 'technology + art' integrated business closed-loop; in the long term, we will focus on essentially improving technical capabilities—using the refinement of playing skills to drive breakthroughs in robot motion accuracy and environmental adaptability.

Ultimately, the team aims to create truly human-like dexterous hand and robotic arm systems, with intelligence and flexibility comparable to humans. These systems can be applied not only to high-precision operations in industrial production but also seamlessly integrated into household services, thus becoming a revolutionary technology that changes human lifestyles." 

The performance of this robot band is a microcosm of how motion capture technology empowers the intelligent evolution of robots. As robots' replication of human movements advances from industrial-level precision to humanistic-level delicacy, with the help of CHINGMU's optical motion capture system, robots can conduct comprehensive motion performance evaluations through posture data in various dimensions such as joint angles, postures, speeds, and stability during training. This will provide developmental support for more robot application fields such as rehabilitation medicine, precision manufacturing, and high-risk rescue. 

From a piece played on stage to diversified industrial applications, motion capture technology is redefining the relationship between humans and machines. In the future, CHINGMU will continue to expand the application boundaries of optical motion capture, bringing more robots with both precision and scenario adaptability into daily life, truly realizing technology empowerment to make machines understand humans better.