LogInMembers of Theia Interactive Team Max Sims and Sean Mitchell talked about their biometric tool called Claria and explained how it leverages VR and Unreal Engine to get the honest truth of a subject’s biological reactions.
Introduction
Max Sims, Senior Product Manager: I’m the Senior Product Manager at Theia Interactive for Claria, a VR bioanalytics insight tool. I came to this position with a background in design, beginning from the moment I discovered that CG was a viable career while in school at the University of Illinois. I continued developing my interests at Art Center College of Design and California College of the Arts. I went on to both practice design and teach others before coming to Theia Interactive.
Sean Mitchell, CTO: I’m the CTO at Theia Interactive, where I am presently leading several research and development initiatives throughout the company. I’m an entrepreneur with a wide range of experience, from large-scale technology-assisted interactive art installations to blockchain to SaaS development to simple e-commerce sites. I co-founded Verdiseno, Inc., which is the company behind SolarDesignTool.com and LyraSolar.com. Additionally, I co-founded a successful makerspace in California with two locations. Here at Theia, I am responsible for the technical direction and various technologies deployed in the Claria stack.
The Claria Team
Max Sims: As the Product Manager, I am responsible for defining the vision for the Claria product, while Sean is helping to define our technical direction and strategy. We also have a great team helping us bring the technology to market.
Sean Mitchell: Our team of artists is responsible for building and testing demo content during the development phases, and another team helps out with technical artistry, database engineering, full-stack development, and dev ops. We operate in several time zones across North America and Europe. We’re currently putting a stronger emphasis on Claria’s backend, primarily with concern to facilitating administration, data, and machine learning.
Claria and its Capabilities
Max Sims: In short, Claria empowers product designers, architects, and engineers to engage their target market and obtain valuable feedback in the form of bioanalytics. Leveraging Unreal Engine and the HP Omnicept, users can arrive at a better understanding of how subjects experience certain objects or environments in VR before they jump into potentially costly manufacturing.
It can get the honest truth of a subject’s biological reactions. By tracking six different biometrics, a designer can justify their design based on how a user would react and learn a design, rather than asking questions and hoping the user can articulate a response. This is a fundamentally new way to design. It takes in human factors, psychology, and neuroscience to truly know how someone feels about an experience.
Sean Mitchell: An experience is created for Claria using our Claria SDK. Once packaged, the “Claria Experience” is then loaded into the Claria VR Client and the subject starts a new session. Throughout the session, the Claria VR Client samples data from the Omnicept’s sensors at a target sample rate of 90hz, and continuously broadcasts these data points to the Claria backend. The backend hosts an administration Interface that enables the management of data, the review of sessions, analysis of aggregates, comparisons of design variants, and several other unannounced features still under research and development. Additionally, any individual session can be played back and reviewed for further analysis inside of the Claria VR Client. During playback, various data visualizations can be found, such as a heatmap overlay that illustrates the subject’s gaze and focus in 3D space.
Max Sims: Claria works exclusively with the HP Reverb G2 Omnicept Edition headset. The Omnicept has an inside-out tracking system, as well as a 2160 pixel display (per eye), which is great for providing high fidelity design previews. Unreal Engine is a big component of Claria too and delivers the highest quality graphics and response times.
Sean Mitchell: Claria was created to help designers understand their designs and how their target market might think and eventually feel about their design proposals. This can be a medical situation or a human-machine interface (HMI) – like operating a product, car, or plane, for example. What makes Claria unique is that it uses the HP Reverb G2 Omnicept Edition VR headset that can read biosignals like eye gaze, pupillometry, facial expression, and heart rate. We can set users a task to accomplish and see what their cognitive load is when they are trying to complete the task in a certain amount of time. You use muscle memory, working memory, and mental exertion to “get the job done,” and that can be tracked. From this, a designer can prove how easily their design is to use and learn. Previously, the subject of a design evaluation would have to self-report and often can not articulate their true point of view. With Claira, the signals are automatic.
Choosing the Headset
Max Sims: We started working with HP more than a year before the introduction of the HP Reverb Omnicept, and it’s been a strong relationship on many levels. HP has invested in Theia through their venture fund, for example, and we were one of the very first independent software vendors (ISV) to be presented with the technology. Beyond that relationship, we had similar aspirations for what VR could do from a biometrics standpoint. No other VR headset had the same level of sensors, especially the EMG sensors that analyze facial muscles to the heart rate. The ability to easily and accurately correlate pupil dilation to heart rate variability from a single device was a big factor in our decision to support the HP Omnicept in the early versions of Claria.
Sean Mitchell: The Omnicept offers a range of possibilities created by the various sensors that nothing else can match. It provides an excellent sample rate of these values, and the resulting data made available through HP’s provided SDK is consistent.
We are exploring the feasibility and convenience of partial support for other HMDs with Claria. At this time, no other HMD provides feature parity with the HP Omnicept. We will continue to evaluate equipment as more hardware hits the market.
Unreal Engine 5 Plans
Max Sims: UE 5 is still a year away, and initially it will be very game development focused, but the examples they have been previewing are mind-blowing. When UE 5 is ready for major OEMs (Original Equipment Manufacturers) for enterprise, we will jump right in!
Sean Mitchell: UE5 was still just a twinkle in Tim Sweeney’s eye when Claria entered the planning stage. That said, we’re going to begin testing on UE5 in the next month or so. Given the relatively straightforward upgrade path from UE 4.27 to UE 5.0, I do not foresee any complications at this time.
Future Plans
Max Sims: We want to make Claria even better at processing the data, and make it more flexible and customizable for our customers and their needs. We also want to make it so the customers don’t need an Unreal Engine developer to get an experiment up and running. Although many students coming out of design school use UE, any designer should have access to the power and insights that Claria provides.
Sean Mitchell: Meeting the technical requirements for the enterprise audience is important for us. There are many parameters to consider such as network restrictions, data integrity, data replication, data management, third-party integrations, and various reporting capabilities. Claria has more layers to it than the Claria Client based on Unreal Engine, and it’s up to my team to ensure that the other technology in the stack not only contributes towards a great Claria VR experience but maximizes the potential of data resulting from those experiences. What is available in Early Access artificially limits Claria’s capabilities while enabling continued innovation with minimal risk for our coveted early access customers.
Max Sims: The future of biometric tools is very promising, and it should help companies make much better-informed decisions, rather than just relying on the HIPPO (the “Highest Paid Person’s Opinion”). The numbers will back up the designer. Luckily, HP has taken great care to conform to General Data Protection Regulation (GDPR) and employs data anonymization in its machine learning algorithm. Claria users will have to conform to Independent Review Boards (IRB) for ethical standards. It is similar to American HIPPA laws, which control the privacy of individuals' private medical information. Claria has put in controls to prevent the biometric information from being abused, but our customers are also adding to that.
Conclusion
Max Sims: We are seeing immense interest from the automotive industry, which wants to know more about a driver’s experiences and responses with different elements of a vehicle. We are also seeing interest from companies in the aerospace and architecture industries for training, wayfinding, and more.
Claria helps ensure designers are putting their best work forward in confidence. With Claria, users can also better justify their design with numbers, just like sales and marketing professionals might. This helps empower designers and companies by showing just how much good design matters and how powerful it is in shaping our human interactions.
Any industry that has machine and man interfaces like manufactured consumer products, aerospace, architecture, and advertising would benefit from using Claria. Where these industries were relying on focus groups and self-reporting to understand how someone thinks and feels, we can now cut the guesswork and arrive at more informed and objective answers.