LogIn
BTS band members performed with Coldplay as holograms in augmented reality.
Coldplay and BTS performed “My Universe” on NBC’s The Voice together in mixed reality, with Coldplay performing in person and the BTS band members performing as Holograms in Augmented Reality, using 3D virtual avatars created via volumetric capture, and rendered live using Unreal Engine 4.27.1. The volumetric capture assets of BTS with holographic treatment were provided by Dimension Studio, in 2 formats, an Alembic sequence, and a special mp4 export using Microsoft’s SVF Plugin for Unreal Engine 4.
There were also some technical challenges to overcome with the mp4 approach – the current SVF plugin for UE4 did not have the ability to track the mp4 recordings properly to timecode. This required collaboration between All of it Now and Microsoft to rewrite elements of the SVF plugin code so that the BTS performers remained in sync with Coldplay on stage.
Another challenge with using the original volumetric capture recordings was that the original BTS performances were recorded at 24 FPS to match the music video frame rate, but the Voice is produced and broadcast at 29.97 FPS. This created some lip-sync issues, due to frame blending 24 FPS into a 29.97 output, but All of it Now was able to clean up the lip sync issues in post-production.
"The convergence of volumetric capture and real-time Virtual Production is revolutionizing the creation of broadcast and filmed entertainment content. The Coldplay x BTS 'My Universe' performance on The Voice highlights volumetric capture as a creative technology ready for primetime. We’re delighted to collaborate with the Coldplay, BTS, and AOIN teams to produce this exciting world-class mixed reality music performance." says Simon Windsor, Co-CEO at Dimension Studio.
Don't forget to join our new Reddit page, our new Telegram channel, follow us on Instagram and Twitter, where we are sharing breakdowns, the latest news, awesome artworks, and more.