Cubic Motion paves the way for a new era of digital humans with the launch of Persona
The very same technology that helped power Siren – the world’s first digital human – is now available as a product for companies to bring in-house. Cubic Motion, creators of true-to-life digital humans, have today launched Persona, a new facial animation product that enables actors to perform as high-fidelity digital characters in real-time.
Persona continually captures and translates an actor’s performance onto a computer-generated counterpart, enabling immediate animation in real-time engines such as Epic’s Unreal 4. Persona paves the way for a new era in digital performance and virtual production. An era where characters, actors and audiences all co-exist and interact.
Steve Caulkin, CTO at Cubic Motion, comments – “Our advanced computer vision techniques allow Persona to capture an actor’s performance with stunning accuracy and detail. Resulting animated characters are able to respond and engage with real-world events immediately. Whether it’s for immediate broadcast, live installations, virtual production or offline animation, Persona will provide you with faithful replication of the actor’s performance.”
The use cases for Persona are virtually endless, from bringing game and movie characters onto the stage at live events, allowing digital characters and avatars to engage with fans in real-time online conversations, to providing the perfect pre-visualisation solution for any virtual production.
Cubic Motion’s computer vision technology has already proven successful at a host of industry events, including GDC and FMX, and was the winner of SIGGRAPH 2016’s Real-Time Live showcase.
HMC
Persona’s helmet-mounted camera system (HMC) and on-body animation system facilitate a completely untethered performance. The proprietary design is capable of delivering consistent, accurate capture over prolonged periods of time.
Using Cubic Motion’s signature front-facing and profile camera configuration – as well as adjustable, inbuilt infrared lighting – the HMC can accurately record even the most subtle facial expressions and poses. The onboard timecode electronics guarantee controlled motion capture, in-sync with all other components of production on a shoot.
Performance Pipeline
At the heart of each Persona setup is an on-body computing appliance called the ‘Performance Pipeline’, which captures and solves facial performance onto a CG character. The Performance Pipeline is a lightweight, battery-powered piece of hardware. It can be tethered or wireless depending on the user’s needs and comes with a flexible harness system, allowing actors to maintain freedom of movement.
To ensure faithful representation, the ‘Performance Pipeline’ is configured to suit each actor and the character they portray. When combined with Cubic Motion’s Identity System, an engaging and intimate portrayal of the character is delivered. The pipeline also provides full capture capability, meaning that all video and animation data can be reused after live events or polished before the final broadcast.
Identity System
Cubic Motion’s ‘Identity System’ uses machine learning algorithms to match a purpose-built tracking model for the actor with a bespoke solver for the character rig – all in a form that is ready to be consumed by Persona. By analysing both the actor’s facial features and capabilities of the character rig, Persona can interpret the unique qualities of this pairing and more accurately transfer a performance into the digital world.
Performance Manager
Persona provides a GUI application called the ‘Performance Manager’, which allows users to remotely monitor each actor and configure their performance. After installing the application, actors are automatically detected and tracked. An array of tools are provided by the Performance Manager, simplifying everything from the alignment of the HMC to control over the animation stream. Users can even preview character animation before it is rendered.
About Cubic Motion
Cubic Motion is the world leader in real-time computer vision and digital animation, renowned for its achievements in performance-driven facial capture. The company’s technology and processes are developed by a team of award-winning PhD scientists, animation specialists, and producers. Cubic Motion licenses products and services for use in video games, film, mixed reality and live events, to name just a few sectors. Recent credits include Siren, Spider-Man, God of War (2018), Metro Exodus and Call of Duty: Black Ops 4.