Motion capture — or “performance capture”, the more actor-friendly term that has only recently entered mainstream awareness — is now standard practice for modern blockbusters. Today, we barely blink at fully rendered CG characters. They are so ubiquitous that all mystery surrounding their creation has faded. As evidenced by the flawless, lifelike animation we see on screen today, motion capture has advanced at a breathtaking pace over the past few decades. Fittingly, it reached previously unimaginable heights in Dawn of the Planet of the Apes, where entire ape armies were animated using this technology. But how did motion capture begin, and which films and filmmakers paved the way for it?

You might picture motion capture as ping-pong balls and ridiculous lycra suits. Well, you are not wrong. In fact, foundational studies of human movement date back to the 19th century, predating cinema itself, primarily to better understand how humans and animals move. Before Hollywood adopted it in the late 20th century, motion capture was a photogrammetry analysis tool in biomechanics, used for sports, education and even video games such as Prince of Persia.

Here is the basic principle of motion capture: real performers provide templates of human movement, and those movements are translated into another medium. One of the earliest entertainment uses of motion capture can be found in animated shorts by animator Max Fleischer. In 1915, he invented the Rotoscope, a concept allowing animators to trace over frames of live-action footage, giving cartoon characters recognisable fluidity, if not photorealistic appearances. Fleischer’s series Out of the Inkwell used this technique, featuring classic animations including Koko the Clown and Fitz the Dog.

Motion capture, Prince of Persia style. Walt Disney liked what he saw. Searching for fresh, exciting ways to bring characters to life, Disney adopted rotoscoping (also known as live-action reference) and began experimenting with it for films. In 1937, the studio released its first feature-length film, Snow White and the Seven Dwarfs. Its animation relied on footage of actors performing the roles, whose movements were then traced. It became a smash hit, and Disney reused rotoscoping for several other features, including Peter Pan, Alice in Wonderland and Sleeping Beauty.

Motion capture did not make massive leaps forward in the decades that followed, even though rotoscoping never truly fell out of use. In 1978, Ralph Bakshi used the technique to animate his truncated The Lord of the Rings film, and later produced American Pop (1981) and Cool World (1992), starring Brad Pitt opposite the animated Kim Basinger. In the 21st century, director Richard Linklater brought back rotoscoping for his experimental drama Waking Life in 2001, where every reference frame was digitally animated, and once again for A Scanner Darkly in 2006. By this point, however, motion capture had found new life.

You may not have heard of Sinbad: Beyond the Veil of Mists, yet it marks a major milestone in animated film history: it was the first feature film created primarily using motion capture. While the movie did use traditional animation in places, most character movements were captured using pioneering 3D optical technology. Actors performed their roles covered in dozens of tiny reference markers; cameras tracked the markers, and computers triangulated their movements. By the time hundreds of Indian animators finished the film for release in 2000, the technology was already becoming known (the domestic box office gross was less than $300,000), but it blazed a trail for future cinema.

Hollywood quickly embraced optical motion capture and the results it delivered. MoCap was expensive and prohibitive, and saw limited use at first. Arnold Vosloo wore tracking lights for certain VFX shots in The Mummy (“Many times I walked around set looking like a Christmas tree,” he recalled). Ridley Scott captured motion data for thousands of extras in Gladiator, mass-replicating them to create realistic crowd scenes. Motion capture was fast becoming an essential tool in the arsenal of forward-thinking filmmakers. George Lucas took a gamble on motion capture in 1999, with Ahmed Best performing the motion for Jar Jar Binks in Star Wars: Episode I – The Phantom Menace, wearing a “tight wetsuit” over which animators drew the Gungan character. The first fully digital lead character in cinema history also turned out to be one of the most reviled film characters ever, yet that did not change the fact that motion capture was rapidly becoming indispensable for visionary directors.

Arnold Vosloo in The Mummy. You have to hand it to the creators of Final Fantasy: The Spirits Within. The whole industry watched Square Pictures to see whether their photorealistic, motion-captured computer-animated epic would succeed or fail. Sadly, it was the latter. Though Square Pictures claimed they could forever change the face of cinema, with “CG actors” such as Aki Ross — whom Square planned to cast in further films — Final Fantasy: The Spirits Within crashed and burned in 2001, costing the company $137 million. Even so, it remains a benchmark film in the evolution of motion capture: 1,327 live-action shots were filmed and later mapped onto digital animation. Even the film’s harshest critics had to admit the visuals were stunning. But the best was still to come.

That same year, Peter Jackson teased Gollum’s appearance in his first Lord of the Rings film, The Fellowship of the Ring. By the time Gollum was ready for his full feature debut in 2002, Jackson had pushed motion capture to a new level: The Two Towers was the first film to use real-time, full performance capture. Actors — still wearing the season’s unflattering lycra suits dotted with ping-pong balls — could perform on set, outdoors and alongside other actors in the moment.

This key distinction was transformative: actors no longer needed to deliver motion capture separately in a remote studio after principal photography. They could inhabit their characters as if they were flesh and blood. Andy Serkis’s performance was part of what made the Lord of the Rings films so highly regarded, alongside WETA’s mastery of the technology.

Serkis would of course go on to pioneer the craft, co-founding his own studio, The Imaginarium, and dominating roles requiring humanised CG characters. He later reunited with Jackson to bring King Kong to life. Meanwhile, other filmmakers began to realise the boundless benefits of MoCap. Ang Lee even put on a lycra suit to deliver selected motion capture sequences for the Hulk. Suddenly, the technique seemed far less ridiculous.

Andy Serkis as Gollum in The Two Towers. Director Robert Zemeckis seized the idea and ran with it — in fact, he ran so far that he lost sight of traditional filmmaking. Where Final Fantasy: The Spirits Within failed, he succeeded. His Christmas fable The Polar Express was filmed using the new performance capture technology and became a hit, boosted by IMAX 3D screenings that set box-office records. However, subsequent films from Zemeckis’s studio ImageMovers received less enthusiastic responses from audiences: Beowulf underperformed, and Jim Carrey’s A Christmas Carol could not replicate The Polar Express’s success. When Mars Needs Moms flopped, Zemeckis scrapped his proposed animated Beatles vehicle Yellow Submarine and quietly shut down ImageMovers permanently.

Many believe the poor audience reception to Zemeckis’s later films stemmed from the so-called Uncanny Valley — the effect that occurs when animated figures look almost human… yet not quite human enough. The brain is tricked by photorealistic visuals and recognisable movement, but a small detail breaks the illusion, making the imagery unsettling to viewers. (Hint: it is almost always the “dead” eyes.)

While photorealistic animation fell out of favour for a time, motion capture found good use in stylised cartoon animation. In 2006, two out of the three Oscar nominees for Best Animated Feature used motion capture: winner Happy Feet and Zemeckis-produced Monster House were MoCapped, while Pixar’s Cars was not. Pixar was never a big fan of the technology, preferring to let its animators rely on intuition to guide their art rather than raw data. The credits for 2007’s Ratatouille proudly declared “100% pure animation — no motion capture!”

As so often proven, James Cameron changed the game. The director waited more than a decade for technology to catch up with his vision. Cameron pioneered a new system he called the “virtual camera”, an entirely new way of shooting. While actors delivered performances and “performance capture” in Cameron’s custom MoCap volume, nicknamed “The Vault”, the director could see their CG counterparts live on his monitors, inhabiting the fully digital world of Pandora. Now you did not even need physical sets outside the capture volume. Cameron called the virtual camera “a pure form of creation”. Even Steven Spielberg was blown away by the technology and later adopted it for his own animated feature The Adventures of Tintin. Blending the best parts of live-action performance and digital artistry, James Cameron’s Avatar went on to earn more than any other film in box-office history.

Sigourney Weaver and Sam Worthington begin their motion capture work for Avatar. Still, the technology truly came of age with 2011’s Rise of the Planet of the Apes. The ape character Caesar — arguably the film’s lead — represented the most ambitious use of the technology up to that point. With Andy Serkis fully embodying the primate role, cutting-edge technology extracted every nuance from his performance. The Planet of the Apes prequel came closer than ever to achieving the seemingly impossible: it almost makes you forget you are watching a construct of ones and zeros. Serkis’s portrayal of Caesar was so well received that calls were made for Academy Awards to formally recognise motion capture performances.

Serkis remains a major figure in motion capture circles, returning to the role of Caesar this month in Dawn of the Planet of the Apes. He also played Godzilla in Gareth Edwards’s reboot and took unspecified roles in blockbusters such as Avengers: Age of Ultron and Star Wars: Episode VII. Performance capture is now a widely accepted method and has become standard. Even odd examples such as Benedict Cumberbatch (very much bipedal) delivering MoCap for Smaug (definitely a dragon) in the second Hobbit film attract little notice.

Performance capture has finally hit the sweet spot for delivering authentic human input to animated characters. Once upon a time, Hollywood actors feared they might one day be replaced by fully compliant digital doubles who could be made to sing, dance and sell anything from behind a computer. Modern motion capture proves that may well be possible one day, yet there is a silver lining: if actors want the work, there will still be jobs wearing suits covered in ping-pong markers.