Piotr Uzarowicz, Head of Partnerships & Marketing for Arcturus, shared some information about HoloSuite and talked about its features.
Introduction
Arcturus is the global leader in the authoring, editing, and distribution of volumetric videos. We were founded in 2016 by a collection of some of the best and brightest from Autodesk, Dreamworks, Google, Pixar, YouTube, and more. We came together and created HoloSuite, the first post-production platform for editing & distributing volumetric video.
Volumetric video is transforming the face of media creation and consumption, and HoloSuite is the foremost platform for the editing and streaming of volumetric video. Our distribution platform provides an adaptive streaming solution that extends the same technology utilized by streaming giants like Netflix, Amazon, and Disney, to stream 3D content directly to the viewer.
My name is Piotr Uzarowicz, Head of Partnerships & Marketing for Arcturus.
Working with DOCOMO
To bring volumetric videos to mobile devices, DOCOMO is leveraging its 5G network speeds and CDN expertise with Arcturus’s compression tools, adaptive bitrate streaming solution, and our video player capable of adapting to a device’s bandwidth in order to maintain the highest quality possible. Utilizing these tools, playback length in volumetric videos is no longer an issue. Users watching on a web browser or experiencing them in AR or VR, can now explore travel hotspots, visit retail locations, watch full workout videos, and more, all from their chosen point-of-view, all on their mobile devices.
Volumetric Videos and HoloStream
Volumetric video records video in 3D, capturing the object or space three-dimensionally in real-time. Volumetrically captured objects, environments, and living beings can be transferred to the web, mobile or virtual worlds to be viewed naturally in 3D. Content is captured by an array of cameras that surround a subject, then software algorithms process the captured imagery to derive a 3D animated model.
These holograms are fully three-dimensional, making them a natural fit for any spatial experience, including AR and VR, and also expand the creative possibilities of 2D mediums. They can be incorporated directly into virtual productions using real-time camera tracking and can be placed into 2D compositions to act as a base for visual effects processing, or to creatively adjust a subject’s performance all the way through post-production.
HoloStream is an adaptive bitrate volumetric video streaming solution. It accomplishes a lot in a few steps. It starts with encoding the OMS files (HoloEdit’s unique file format) into several different quality profiles, ranging from low to high, so depending on the user’s Internet connection and their device they will receive the best quality possible. Just like with Netflix, when moving from one internet connection to a better or worse one, the stream adapts to that connection. The additional feature of HoloStream is that it streams polygons rather than pixels, which enables interactivity with the volumetric capture.
Implementing the Toolkit
It’s straightforward and can take no more than a few minutes to add streaming volumetric videos to a web page or game.
First, a couple of basic definitions. A volumetric stream is described by a media presentation description (MPD). This is an industry-standard file format for defining a stream. It contains all the information needed to describe the stream, including the different quality profiles and their associated bit rates. Each volumetric clip on our server has a unique MPD file that is stored at a unique URL.
To add volumetric video streaming to an application, you create an instance of our streaming player plugin, provide the player the URL for the volumetric clip that you want to play, and tell the plugin instance to start playing.
At any stage, you can change the URL of the clip being played, start/stop playback, or alter the camera view through the viewport. The process is slightly different for each development platform, but in a nutshell, if you are familiar with the platform you can add volumetric video streaming support in under 5 minutes.
Editing Volumetric Videos with HoloEdit
With HoloEdit we have made fundamental advancements in the editing of volumetric videos, bringing elements of interactivity that were not possible without our platform. That includes dynamic interaction and branching narratives.
HoloEdit advances:
- Non-linear editing of performances
- Auto-Skinning and Rigging
- Author Real-time Interactions
- Geometry Editing & Cleanup
- Texture Editing & Cleanup
- Temporal Compression
Challenges and Limitations
While volumetric video has been developing rapidly over the last few years, it is still relatively unknown to creators worldwide.
Awareness and education are the biggest issues volumetric video currently faces. I can’t wait to see it in use everywhere, and I will in the next few years. Right now though, we need more creators exploring and experimenting with the format.
Use cases run the gamut, and there is a huge amount of potential. For example, in the gaming industry, we’ve seen games evolve and get more realistic with every release, but they are still a little in the uncanny valley. Imagine incorporating photo-real captures into video games… the immersive quality of volumetric videos working hand-in-hand with the improvements that game engines have made may power the evolution of video games.
Future of Volumetric Videos
Volumetric video lets users consume entertainment content in new ways. From viewing live sports in holographic form, interactive digital human assistants, e-commerce (where buyers can view products in 3D), virtual performances, and more. In the near future, gaming, movies, and marketing will all be disrupted by integrations of volumetric video content. Imagine hologram hotel concierges, virtual nurses, and hologram salespeople of all types, and you’re just beginning to scratch the surface