LogIn
Introduction
Nex was started in 2017 by former Apple engineers and was invited to join Apple on stage for the launch of the iPhone XS, which featured the A12 Bionic with a neural engine. Nex built an app called HomeCourt that uses advanced AI and motion tracking on mobile devices to analyze basketball movements. The NBA invested in Nex to further develop HomeCourt.
In 2020, HomeCourt went viral during the pandemic as youth sports were canceled and many athletes used it at home to continue training. During that time, Nex worked to integrate motion-tracking AI with Unity so that digital-physical activities could be even more video-game-like. Then Nex built Active Arcade, which became a worldwide hit in 2021, engaging more than 3M users worldwide and becoming the #1 fitness app at that time in Hong Kong, Taiwan, Thailand, and Malaysia. From there Nex has continued to build even more robust motion games and developed tools that any game developer can use to build motion-based games in Unity.
The Story of Nex
While Nex started applying its AI motion technology in sports training, we quickly realized that the more we gamified our training activities, the more our users engaged. That led us to experiment with merging sports and video game elements. The feedback from these sports-themed motion games was overwhelmingly positive, creating even more joy and engagement for users around the world.
By merging sports-like movement with interactive content created in Unity, Nex is leading a new category of interactive experiences. Instead of requiring expensive headsets or sensors, Nex motion experiences work on any modern camera-enabled device. There are no headsets to put on, controllers to learn, or sensors to wear. Nex can track 18 points on your body and map the movement to the gameplay in real-time.
Motion Games
Motion games remain incredibly popular. Nintendo Wii led the category in the early 2000s and sold more than 100M consoles. Its family-friendly content unlocked huge growth beyond traditional gamers and tapped into casual consumers. Nintendo has continued this with Switch which has now eclipsed Wii sales figures, with its Joy-Con controllers.
Meta Quest is also a “motion-enabled” experience, using cameras in the headset and motion-sensing controllers to power the gameplay. Beat Saber, the motion-based music game, has been a breakout hit on Meta Quest. However, its audience has been limited to VR, which can be expensive and inaccessible to many audiences. Nex believes that motion experiences can be accessible to everyone, played by people of all ages, in the living room or on mobile screens – unlocking a new, even larger addressable market for motion entertainment.
The Role of AI
Nex has built a custom neural network for motion tracking that uses machine learning to achieve high levels of accuracy and performance. Motion games on consoles like Wii pre-dated the development of AI and machine learning, so motion tracking was often crude, under-performant, and not highly accurate.
The other key development has been the integration of the Nex MDK into Unity, so game developers don’t need custom tools or knowledge of AI models to utilize motion tracking in their game development process. This makes it easy for any game developer to create these motion-powered experiences using tools they already know.
Utilizing Smart TVs
As Smart TVs are starting to “catch up” to mobile computing, they will be able to run richer interactive applications and games. In many ways, smartphones and tablets are already starting to compete with game consoles in terms of graphics and performance. This level of computing power is now coming to streaming boxes and Smart TVs.
The biggest barrier to gaming on these devices has been the lack of a natural input mechanism. TV remotes are poor input devices for games, and game controllers can be intimidating accessories for casual consumers and families in the living room. Nex believes that natural and intuitive motion-based interfaces will unlock these experiences in the same way that touchscreens unlocked mobile use cases.
Nex Motion Tracking Tools for Unity
The Nex MDK is a simple AI-powered toolkit for Unity. Right within Unity, developers can use action recognition detectors that are pre-built by Nex to detect specific motions. For example, swinging your arm, jumping up and down, or moving laterally are all standard actions that can be detected automatically simply by selecting them in a drop-down menu.
Game Designers can map 3D avatars and character movements using points of the body such as the shoulder, elbow, wrist, hip, or knee. With these tools, it becomes easy for any game designer or developer to create motion game experiences. The Nex MDK is optimized for performance on a wide variety of devices, so developers don’t need to do extensive testing and analysis on different mobile devices, PCs, or Smart TVs that they might be targeting.
Use Cases
One example is how we’ve worked with Hasbro on a title called Peppa Pig: Jump & Giggle. We worked with their interactive studio (eOne) to map the IP and experience of Peppa Pig TV episodes and merge it with active, motion-powered gameplay using the Nex MDK. This created a joyful experience for kids and families, where they can jump in muddy puddles with Peppa and their friends. With this new game from Nex and Hasbro, kids can have a “virtual playdate” with Peppa and interact with the brand at a much deeper level.
Another example is a game studio in New Zealand called cookiecrayon. They are an indie game studio that has made award-winning games for mobile and PC, with unique game design and artistry. Nex connected with them to collaborate on a motion game, and they started using the Nex MDK right away. Since then we’ve been working in a relatively hands-off role. They asked a few questions and started using the MDK right away with no additional training or support necessary. The result was a demo of a new motion game called Neo-Witch that they produced in about a two-week development sprint. It’s a spell-casting game that makes you feel like a magician or wizard, casting spells just by using your arm and hand movements (like Dr. Strange!)
Current Plans
We are preparing to launch a motion entertainment system for TVs called Nex Playground later this year. While our technology enables games across smartphones, tablets, and PCs, we’ve heard from our users that they want our content on the biggest screen TV in their homes.
While we are also working with TV manufacturers like Sony and Samsung to integrate our tech, those next-generation TVs can often take years for consumers to adopt. Nex Playground will bridge that gap and make it possible to play motion games on any existing TV. It’s a low-cost device that is built for families to play motion games together in their living rooms. With this device, we will also be launching a subscription service, so consumers can get an endless stream of motion games and activities, and developers can participate in an expanding catalog of game content.
We are currently working with developers who want to participate in the launch of Nex Playground and the subscription service, the target launch timeframe is this Christmas, December 2023. We will have more details about the system later this Summer, but consumers and developers can sign up for early access here.