How Motion Capture Brings Picture Books to Life

Technological Innovations in Children’s Education: How Motion Capture Brings Picture Books to Life

Clients: Shanghai Film Art Academy, Zhonghua Vocational School, Deng Xin Picture Book Studio

Applications: Motion Capture, AIGC Picture Book Creation, Children’s Publications, Digital Education

In recent years, the integration of AI, VR, and other technologies has revolutionized various aspects of our lives, including children’s education. Traditional picture books are evolving into dynamic experiences that combine animation, sound, and video. This transformation breathes life into static visuals and text, enriching storytelling, boosting children’s interest in reading, and enhancing knowledge absorption.

A Milestone in Digital Picture Book Creation

In the field of digital picture books, motion capture technology is redefining traditional workflows with innovative applications. The Jinglang Huaguang Zhiyi Digital Picture Book Studio, jointly established by Shanghai Film Art Academy, Zhonghua Vocational School, and Deng Xin Picture Book Studio, has collaborated with CHINGMU to advance industry-academia-research integration.

By leveraging motion capture technology and integrating it with CG production, 3D rendering-to-2D, and virtual reality—technologies commonly used in the animation and film industry—the studio has created a groundbreaking AIGC picture book workflow. To date, the studio has published 38 picture books, with a total circulation exceeding 1 million copies.

Some notable works include:

•Beautiful China: Building Wisdom

•My National Treasure Files

•My Chinese Space Class

•Leadership: Stories of Shanghai

•Remarkable National Heavy Equipment

•Panda Astronaut

These books cover diverse topics, such as the history of China’s aerospace industry, Shanghai’s urban development, and the ingenuity behind national engineering achievements.

Among these, My Chinese Space Class has gained significant attention and popularity. The book’s protagonist, a panda, educates readers about China’s aerospace history through engaging and interactive storytelling. Similar to CCTV’s latest AI-powered virtual panda, this character is both adorable and technologically advanced, delivering knowledge with charm and credibility.

During the panda’s design phase, the studio collaborated with CHINGMU to enhance its realism. Optical motion capture technology was employed to transfer the nuanced expressions and gestures of real performers onto the panda character in real-time. This process brought the panda to life, creating a vivid and relatable persona. The combination of lifelike design and extensive knowledge makes the panda an engaging medium for immersive and interactive education.

Efficiency and Reusability in Picture Book Creation

According to Bao Wenjun, Director of the Digital Entertainment Center at Shanghai Film Art Academy, the adoption of motion capture and other advanced technologies has increased picture book production efficiency by nearly 50%. Additionally, the flexibility of digital technology enables the creation of reusable digital assets, greatly facilitating the development of derivative products and multimedia promotional materials for picture books.

Reported by Suzhou Daily

Media Recognition and Public Engagement

Thanks to its innovative creation methods, My Chinese Space Class was invited to participate in the 14th Jiangsu Book Fair and received extensive media coverage, including from Suzhou Daily.

At the exhibition, themed “My Chinese Space Class—Metaverse Motion Capture + Immersive VR Experience”, CHINGMU served as a co-organizer alongside Jiangsu Phoenix Science and Technology Publishing House, Deng Xin Picture Book Studio, and the Digital Academy of Shanghai Film Art Academy.

Using CHINGMU’s motion capture system, the panda character was brought to life on-site with real-time animation and rendering. The panda not only interacted with visitors by striking poses and taking pictures but also offered an engaging space exploration experience for both children and adults.

The Future of Digital Picture Books

By incorporating motion capture technology and digital effects, picture books are becoming more dynamic and content-rich. The addition of animated lighting effects further blends learning, visual entertainment, and storytelling into one immersive experience.

This innovative approach not only makes the reading process more enjoyable but also fosters children’s interest in books, stimulating their imagination and creativity.

Key Features:

1.Cutting-Edge Technology: Integration of motion capture and AIGC workflows.

2.Immersive Experience: Real-time interaction and visually rich storytelling.

3.Scalable Solutions: Reusable digital assets for efficient production and derivative product development.

CHINGMU continues to empower digital education and storytelling, transforming how knowledge is presented and experienced.