How does motion capture technology create a new era for games?

Source：

2024 Nov. 22

    Applications: Motion Capture, Triple-A Games, Character Capture, Quadruped Animal Capture, "Black Myth: Wukong", "Mortal Kombat", "Red Dead Redemption 2", Steam.

On August 20th, "Black Myth: Wukong", the first domestic AAA single-player game, immediately went viral across the entire internet once it was launched, dominating the screens of various major platforms! The interaction volume on overseas social media was nearly 100 million, and the domestic whole-network interaction exceeded 180 million. Many media platforms including CCTV News, China National Radio Network, Global Times Online, and Guancha.cn have reported on it! Moreover, "Black Myth" achieved 400 million yuan in pre-sales, with the first-day sales exceeding 1.5 billion yuan, and the number of concurrent users on Steam exceeding 2 million.

The success of "Black Myth: Wukong" is closely related to its creative historical breakthroughs. A AAA game usually refers to a game work with high development costs, a long cycle, and a large consumption of resources. However, "Black Myth: Wukong" completed the production of the game in only four years. In terms of plot design, character shaping, and storylines, it focuses on being original and fully incorporates the Chinese classic mythological story "Journey to the West" into the game. Moreover, in order to make the visual presentation more shocking and give gamers a more immersive experience, the ancient site scenes such as Lingyin Temple in Hangzhou, Dazu Rock Carvings in Chongqing, Xiaoxitian in Shaanxi, and the Tomb Pagoda Forest of Lingyan Temple in Jinan were all scanned on the spot and replicated on a 1:1 scale to ensure the realistic reproduction of the scenes as much as possible.

Meanwhile, in order to enhance the sense of role immersion for gamers and endow them with a more authentic interactive experience, in terms of character shaping, the production team utilized motion capture technology. By capturing the personalized images of real actors, they recorded and collected data information such as facial expressions, movement postures, and martial arts scenes, and incorporated them intact into the creative process of game characters. This makes the performance of game characters more dynamic, allowing every subtle movement, such as running, walking, scratching the head, waving weapons, releasing skills, etc., to be carried out smoothly and seamlessly.

In the motion capture behind-the-scenes footage of "Black Myth: Wukong", the motion capture scene performed by Yin Kai (Kyle), a martial arts student from Shanghai University of Sport, has drawn everyone's attention and sparked discussions. This scene demonstrates how motion capture technology presents every word and deed of game characters, including facial expressions, emotional releases, and physical expressions.

Besides game production, motion capture technology has also shone brightly in the field of film and animation. In 1990, the movie "Total Recall" marked the official debut of motion capture technology on the silver screen. The release of the movie "Avatar" in 2008 further brought motion capture technology into people's vision. Films such as "The Avengers", "The Incredible Hulk", "The Hobbit", "Pirates of the Caribbean", and "Rise of the Planet of the Apes" all used motion capture technology to shape their characters.

In the traditional process of film and animation creation, the shaping of characters relies on modeling technology and subsequent frame-by-frame retouching to create realistic and stunning scenes. However, relying on motion capture technology can minimize the need for manual keyframing. All that is required is to capture the performances of real actors, including their faces, whole bodies, and props, and directly map these real data onto virtual characters. Based on the expressions of real people, the actions of virtual characters appear more natural and realistic, greatly enhancing the audience's sense of immersion and identification.

Relying on motion capture technology, virtual rehearsals can also be achieved, realizing the effect of what you see is what you get. It helps directors, producers, animators and other creative personnel to evaluate the effect of the work in advance, discover problems in a timely manner and make adjustments, thus saving costs and improving efficiency. Especially for the shooting of large-scale productions and big scenes, virtual rehearsals are undoubtedly powerful tools in the creative process. In addition, for some scenes that are difficult to build physically, with the help of motion capture technology, various shooting scenes can be created and various creative ideas can be realized. For example, XR/virtual shooting presents this effect to perfection.

However, for high - quality film and animation masterpieces and AAA - rated games like "Black Myth: Wukong", the requirements for motion - capture technology are even higher. There is a need for greater precision, more capture targets, and diverse capture types.

Whether it is in film and animation or game production, there are always scenes where characters interact with props. For example, in "Black Myth: Wukong", the Monkey King uses the Ruyi Jingu Bang. While the motion capture technology needs to capture the character himself, it also has to capture the prop in his hand. This requires that motion capture is not limited to just capturing the human body, but also needs to simultaneously capture the props in the character's hand to present an interactive performance effect.

In the production process of interactive games such as AR/VR, the hands are the part that can best enable gamers to experience a genuine sense of operation and immersion in the role. However, as one of the most flexible parts of the human body, the hands have diverse movements and are prone to occlusion, thus requiring a high-precision motion capture system for capture. Currently, the Qingtong motion capture system has powerful intelligent algorithms. Coupled with the multi-marker method, it is no longer a difficult task to capture fine finger movements. In addition, the K26, a high-end motion capture camera independently developed by Qingtong , has 2600 pixels and is the motion capture camera with the highest pixels in the world. Its 3D precision can reach 0.02mm. Whether it is a large space of several hundred square meters or the optical finger capture in a multi-person interaction scenario, it can accurately capture and perfectly present every detail and movement expression.

Traditionally, the creation of quadruped animals relied on models or was presented through the method of "using humans to represent animals". Compared with capturing real animals, there might be problems such as a lack of liveliness and stiff movements, and it also placed high demands on actors. Motion capture technology, on the other hand, is based on capturing the animals themselves. By attaching markers to the animals' coats or furs and with the combination of high-precision motion capture cameras and high-performance software, it can reproduce the shapes and postures of the animals on a one-to-one basis, giving the most vivid and authentic feeling to animation production. However, there are great challenges in controlling animals. One is their obedience, and the other is that based on their unique physiological structures, their gaits of movement are completely different from those of humans. Currently, there are very few brands that can achieve quadruped capture. Regarding this issue, after algorithm upgrades and practical tests, Qingtong has finally mastered this technology.

For the fight scenes in large scenes and on a grand scale, by using motion capture technology to capture the wonderful performances of multiple motion capture actors and simultaneously calculate and record them, it can not only present the expressions and movement postures of each character vividly and exquisitely, depict various wonderful scenes, but also significantly reduce the cost of post-production and improve the overall production efficiency.

The role of motion capture technology in film and animation, as well as game production, is not just about the capture technology. The most crucial aspect is its post-processing capabilities, which can minimize the work procedures of animators and enhance the efficiency of post-production. Qingtong post-processing software, CMProcess, can offer you the greatest convenience in post-processing and present the optimal results. This software can automatically repair lost or suboptimal motion capture data with just one click. It includes restoring lost Marker points during motion capture, filling, trimming, and smoothing the original Marker data, performing batch trimming and batch filling of the data, allowing for manual dragging and drawing of the 2D trajectories of Marker points or the 6DOF data trajectories of rigid bodies and skeletons, etc. Moreover, it also supports manual keyframing and saving. It fully takes into account various issues that may arise during the post-processing process.

For high-quality film and animation production, Qingtong has launched a top-notch animation package that can support both the motion capture animation process and the PreViz process simultaneously. It includes hardware and software systems, the K26 motion capture camera with the highest precision in the world, the professional animation creation motion capture software CMAvatar, the integrated recording software CMCapture, the automated post-processing software CMProcess, and is also equipped with an upgraded reference camera. It is fully integrated and comes with a complete set of equipment, which can meet various needs of film and animation customers. It is hoped that with the support of Qingtong top-notch animation package, more top-notch games like "Black Myth: Wukong" will emerge in China, and more high-quality representative film and television works such as "The Wandering Earth" and "Creation of the Gods: Part I" will be produced.

It is hoped that with the support of Qingtong top-notch animation package, more top-notch games like "Black Myth: Wukong" will emerge in China, and more high-quality representative film and television works such as "The Wandering Earth" and "Creation of the Gods: Part I" will be produced.