Using Facebook Augmented Reality in Museums

Introduction

Hello! I’m Sue Young, Director of Product Management and Head of Spark AR at Facebook. 

I always knew I was interested in technology, but I also was always driven by a desire to help people--and that’s where Facebook’s mission spoke to me. I’ve been at Facebook for eight years, which means I’ve gotten to work across a lot of products, including Events, Facebook Marketplace, and Places. When the opportunity to join Facebook Reality Labs came up, I jumped. I was ready for a new challenge and needless to say, AR is a big one! AR is the bridge between mobile and the future, so it just felt like a great chance to stretch my skills and help bring to life a really exciting part of Facebook’s future. 

Possibilities of Facebook AR

At Facebook, we see AR as the next computing platform, one that will change the way we connect and communicate. It has the potential to be the first technology really designed around people. If we succeed, we will see a world where you can defy distance, to feel as though you’re right next to someone even if they’re miles away. It’s a world where our senses are enhanced, like with smart audio that drowns out background noise to help you focus. And it’s a world where intuitive devices let you focus on the experiences around you, not a device at hand. This is all still a few years away, but that’s where Spark AR comes in.

It’s the platform that powers all AR effects and experiences on Facebook--with more than 600 million people using AR across our apps and devices every month. The pandemic has only amplified the ways in which we’ve had to get creative when we communicate, connect, and collaborate. When you can’t be together, and when other 2D tools fall short, AR can create shared experiences and more meaningful moments. It’s already happening today through the experiences we power across Facebook’s apps, from gesture challenges on Instagram to things like Storytime on Portal. This is just the beginning.

Spark AR opens possibilities to creators and brands to contribute to this growing ecosystem (and the opportunity to reach a huge audience). It’s such a flexible, dynamic medium. We see people experimenting with art in their surroundings, and with new modes of self-expression. AR can help people learn about the world around them, and other perspectives. The breadth of possibilities is one of the things I love most about working in this space.

AR Course

We’re dedicated to supporting our global community of creators and are always looking for ways to make AR creation easier and more accessible for as many people as possible. The Spark AR Curriculum is there to equip people, whether they’re new to AR or want to advance say, their graphic design career, with a solid foundation and the opportunity to develop a new skill set. You don’t need to know C++ to create in Spark, though it can certainly help. And we’ve invited people who’ve done it themselves to help shape the courses-- community experts like Ommy Akhigbe and Luke Hurd provide instruction, tips, and tools for creating AR effects, right alongside our own experts.  We’re starting with the Spark AR Foundations course and want to add more advanced courses in the coming year. 

Continued education is just one of the many ways we’re working to invest in AR to create a thriving community and ensure our creators are well-positioned to stay ahead of the curve. We’re here for the long haul.

Tech and Software for AR

It’s been an exciting few years for AR development. There are the full-blown AR engines, the 3D game engines that have added AR, and then dedicated SDKs for mobile. Spark AR, now a few years old, draws from these approaches to simplify and streamline the creation process. Spark actually started as an internal camera platform with our AI team. But today, the full platform is made up of Spark Studio, which is where you add and animate your 3D objects, effects, and patches for interactions and animations.  We have an asset library, or you can of course bring in your own via integrations with Blender and Substance Painter.  And, built right into Spark are a ton of capabilities you can choose to use, from the target and Iris tracking to dynamic music reactivity and render passes for more complex textures and shaders. We added more than 100 new features in 2020 alone. For people who want to jumpstart their projects, we also offer a ton of templates. If you’re interested, definitely check out our blog. 

Of course, the other half of the Spark AR platform is AR Hub, where you submit your effects for review and publish across Facebook, Instagram--and soon, Messenger. Hub is also where you’ll go to monitor your effects and get insights on engagement and share your work with the world (and potential clients) via the Gallery and Portfolio. This full-cycle support of creation is important not only to help with development but with the distribution. 

Future of AR

There are many ways I see AR changing our lives. When you think about it, it’s AI, graphics, and the real world around you converging via the camera in your pocket. These use cases on mobile will help us better understand the demands of AR glasses years down the road. AR glasses will help people move through the world with less friction and more focus on just...being. We really think they’ll tap into and enhance the best of the world around us. 

I think the earliest examples we’re seeing today are a fair proxy for what we can expect in the future. You have museums bringing their 3D collections into AR on Instagram--talk about lowering the barrier of entry. It’s simultaneously preserving history while opening up the opportunity for new people to have a new perspective (and a little fun), with no travel required. Shopping and commerce are another really exciting use case we’re working on. If you can’t try on makeup at Sephora, you can try on and shop some of your favorite brands right on Instagram. AR try-on can increase buyer confidence and make online shopping that much more approachable. Beauty is just the start, we’re working to unlock other verticals soon. 

That’s the awesome thing about AR. It meets people where they are, in a personal and interactive way. 

In the future, I predict every person will be able to create and share AR content as easily as they might a photo today. AR content will play a role in every experience we have, calls we make, videos we watch, and more. The creators who’re starting today will define those future experiences. 

I believe we’re only at the beginning of potential with AR and I’m excited for what’s ahead!

Sue Young, Director of Product Management at Facebook

Interview conducted by Ellie Harisova