Australian basketball player Maddison Rocci stands motionless inside a film studio, with more than 100 cameras trained on every part of her body. She holds her position with arms outstretched as shutters click closed. Behind the lenses sits a crew of filmmakers who normally work with Hollywood directors. Today, however, they are collaborating with scientists.

Once the cameras fall silent, a team of biomechanical engineers and software programmers from Griffith University’s Centre for Biomedical and Rehabilitation Engineering (GCORE), together with film animators from Myriad Studios and Naughty Monkey, uses the data to build Rocci’s digital twin, replicating her anatomy from the inside out.

This is the “digital athlete”. It combines 3D body scanning, MRI scans and motion capture data to generate a detailed representation of body shape, bones, joints, muscles and other soft tissues within Rocci’s everyday performance environment. Scientists can now visualise what happens inside her body as she runs, jumps, twists, pivots and shoots. The loads placed on her muscles and joints are captured, and this data helps coaches design better training programmes or refine technique.

For example, coaches can examine Rocci’s (or any athlete’s) sidestepping motion in real time — a common movement linked to anterior cruciate ligament knee injuries. This information is immediate and personalised, which is critical, because every athlete experiences different loads due to their unique physiology.

Maintaining healthy joint tissue or repairing damaged tissue requires “ideal” loading and tissue strain. Recent research shows this kind of “biofeedback” can be achieved by combining an athlete’s personalised digital twin and motion capture with wireless wearables. These custom digital twins break movements down into smaller, predictable actions, working alongside neuromusculoskeletal rigid-body models, real-time code optimisation and artificial intelligence or machine learning techniques.

Recent work also demonstrates that lab-grade biomechanical measurement and modelling can be performed outside the laboratory using only a small set of wearable sensors or computer vision methods. Commercial, affordable versions of this technology are expected to become available in the near future.

The non-invasive, accurate real-time prediction of internal tissue loads in real-world settings has long been regarded as the holy grail for biomechanists. As this technology advances, training and rehabilitation may soon be guided by biofeedback systems built on digital twins of anyone’s musculoskeletal system.

Real-time visual biofeedback enables people to adjust knee and hip movement as needed, based on their own natural movement patterns or coach guidance. Importantly, when patients effectively modify their movement, they experience clinically meaningful improvements in hip pain and function.

Optimising athletic performance is one application. Creating digital twins in this way carries other potential use cases, including military applications and support for people with disabilities. It may help prevent common musculoskeletal injuries among service personnel and support neurorehabilitation for patients with spinal cord injuries.

A fully integrated system called BioSpine is currently undergoing augmented reality training trials, enabling people with spinal cord injuries to walk within a metaverse or complete physical cycling sessions with electromyostimulation and motion assistance inside immersive augmented reality environments. As the technology evolves, it holds potential to help quadriplegic and paraplegic patients “walk” again.

The authors wish to acknowledge Duncan Jones and Myriad Studios for their significant contributions to this research.