For robots to operate flexibly in the real-world environment, the first step is not to build more sophisticated models, but to acquire sufficient high-quality real demonstration data. Traditional data-collection methods usually rely on laboratory-based setups, including fixed motion-capture systems, limited worktables and strictly-controlled lighting and spaces. While the captured data delivers high precision, it can hardly be transferred directly to real-world sites such as factories, supermarkets, warehouses and households. Worse still, hand-object interaction involves multi-modal information: hand pose, object pose, contact force and visual feedback. Missing data or time-sequence misalignment in any channel will render the collected demonstrations useless for model training.
What really restricts the improvement of robot manipulation capabilities is never algorithmic creativity, but the availability of high-quality data in real scenarios. Jointly developed by CHINGMU and DexRobot, the OptiUMI high-precision portable multi-modal data-acquisition system is designed to solve this practical challenge. It packs professional optical motion-capture precision into wearable hardware, turning every real-world manipulation action into verifiable, trainable and transferable digital assets.
Ⅰ. Key Challenges in Hand-Object Interaction Data Acquisition
The human hand has more than 20 DOF, enabling countless operations ranging from delicate grasping to complex assembly. To allow robots to “observe” and “learn” these motions, data collection must satisfy three requirements simultaneously:
Spatial precision: Millimeter-scale fingertip displacements and tiny object rotations must be recorded accurately.
Time-synchronization: Visual footage, hand poses and tactile feedback must be perfectly aligned; otherwise robots will learn misaligned causal relationships.
Scenario authenticity: Data collected in clean laboratory environments is difficult to generalize to real-world workplaces full of noise and interference.
Existing teleoperation or motion-capture solutions face a dilemma: high-precision systems are bulky and hard-to-deploy, confined to laboratories, while lightweight devices lack the accuracy required for training-grade datasets. For a long time, high-quality hand-object interaction data collection in real-world scenarios has remained a tough problem.
Ⅱ. OptiUMI: Professional Optical Motion-Capture Precision in Wearable Hardware
The OptiUMI system consists of two handheld grippers and one head-mounted motion-tracking camera. Powered by infrared optical tracking technology, it breaks the above-mentioned bottlenecks with three core advantages:
1. Sub-millimeter-level spatial positioning to reconstruct real-world operations anytime, anywhere
The head-mounted infrared binocular camera tracks the 6-DoF poses of grippers and objects in real-time at 120 FPS, delivering sub-millimeter positioning accuracy. Tiny fingertip adjustments and fast-grasping motions can be fully captured without omission. This level of precision was once exclusive to large-scale laboratory motion-capture systems, and is now integrated into a lightweight wearable device weighing only 484 grams. Operators can collect data directly at sorting stations, shelves and workstations, without moving tasks into laboratories.
2. Hardware-enabled multi-modal synchronization for zero-error time-sequence alignment
Hand-object interaction is a multi-physics coupled process. Through hardware-level time-alignment, OptiUMI limits the synchronization error of RGB vision (binocular, 1920×1080@60fps), infrared pose data (binocular, 2048×1536@120fps) and gripper tactile sensing (216 sensing points on each side, 20 N measuring range, ±15%FS accuracy) to ≤ 50 microseconds. Every frame of data received by robots records hand movements, object locations and contact force at the exact same instant — a fundamental guarantee for training stable manipulation policies.
3. Simultaneous hand-object tracking to record full interaction workflows
Unlike solutions that only record hand movements, OptiUMI tracks the 6-DoF poses of both grippers and manipulated objects. It precisely captures relative positional changes, contact moments and separation processes between hands and objects, documenting the complete physical-interaction workflow. This hand-object joint-tracking dataset enables robots not only to imitate hand trajectories, but also to understand how objects are moved, rotated and stressed, so as to master real-world manipulation skills.
Ⅲ. Closed-Loop Pipeline: From Real-World Operations to Training-Ready Datasets
More than a data-collection device, OptiUMI builds a high-speed data channel connecting real-world scenarios and robot hardware.
Instant data capture in real-world environments
Free from large-scale laboratory peripherals, its head-mounted design and lightweight grippers (450 g each) allow operators to collect data on-site in factories, supermarkets, warehouses and other real-world locations. No marker deployment or space calibration required, data acquisition can start immediately after power-on, realizing on-the-job data collection with instant usability.
Dual coordinate system switching, compatible with algorithm research and real robot deployment
The system outputs pose data in two coordinate frames: the ego-centric frame (referenced to the operator’s head), whose data can be transferred across different robot platforms and locations, ideal for policy learning; and the world-centric frame, providing absolute spatial references for simulation reproduction and physical-robot validation. Users can switch freely between the two modes, and one dataset meets multiple research and deployment requirements.
One capture, three use-cases, seamlessly integrated into training pipelines
Automatic data-quality inspection runs during acquisition, eliminating heavy post-processing work. Multi-modal data generated from a single real-world manipulation can be applied to three scenarios:
Simulation platforms: URDF reconstruction and algorithm validation
Physical robot training: Demonstration data for policy learning
Deployment testing: Trajectory reproduction under world coordinates to verify transfer performance
This delivers the concept of One Capture. Three Worlds: a single acquisition session connects the physical world, simulation environments and robot hardware.
Ⅳ. Application Value of Portable High-Precision Hand Motion Capture
CHINGMU’s portable high-precision hand-object interaction data-acquisition technology has been validated in offline scenarios.
At the 2026 World Robot Conference (WRC), CHINGMU officially showcased the OptiUMI high-precision portable multi-modal data-acquisition system, attracting numerous embodied-intelligence research teams and robot manufacturers. As a key product of CHINGMU Vision’s robot-data-infrastructure layout, OptiUMI supports fine grasping, flexible sorting and complex hand-object interaction tasks, and can be rapidly deployed in large-scale scenarios such as data-acquisition factories, supermarket restocking and warehouse picking. During the exhibition, many professional visitors tested the combined solution of head-mounted tracking camera and handheld grippers, and highly recognized its real-time 3D reprojection verification and hand-object synchronous-tracking capabilities.
In scientific-research cooperation, CHINGMU ’s high-precision optical motion-capture systems have been widely adopted in robot-performance evaluation, embodied-intelligence training platforms and full-stack data-services, delivering leading-edge solutions for educational, research and industrial clients worldwide and empowering robot R&D and industrial upgrading.
In the long run, high-precision, wearable, scenario-universal hand-object interaction data acquisition is becoming a critical component of robot-data infrastructure. When data collection is no longer confined to laboratories and every operator can convert real-world operations into training-grade datasets on-demand, robots will learn real-world skills at a dramatically accelerated pace.
Essentially, hand-object interaction data acquisition translates human dexterous manipulation intelligence into readable digital language for robots. Featuring sub-millimeter accuracy, microsecond-level synchronization and joint hand-object tracking, OptiUMI redefines the capability boundary of wearable data-acquisition hardware. It turns high-quality data from a laboratory-exclusive resource into an easily-accessible standard product in real-world scenarios — arguably one of the most important stepping-stones toward general-purpose robotic manipulation.
