Agora's Hermes Frangoudis has told us how the company helps develop the metaverse, spoke about embedding real-time voice and video into games, and shared how to ingrate the company's solutions into game engines.
Introduction
80.lv: Please introduce yourself and your team. Where did you study? What companies have you worked for? What projects have you contributed to?
Hermes Frangoudis: Hi, I’m Hermes Frangoudis Director of Developer Relations and Partner Engineering at Agora. My team focuses on the end-to-end developer experience and being the developer’s voice within the company.
I graduated from Penn State’s College of Information Systems with a focus in Design and Development. Prior to joining Agora, I was an augmented reality engineer at Blippar, leading the NY dev team and building augmented reality experiences for top brands and fortune 500 companies.
Agora
80.lv: Could you tell us about the story of Agora? How did it all begin? What were your original goals?
Hermes Frangoudis: Traditionally, real-time video and voice happen in standalone dedicated applications but Agora recognized that people increasingly want to engage directly in applications they are already using. These real-time voice and video solutions were difficult and expensive to develop, requiring reliable, multipath transmission of substantial amounts of data across many geographies and network operators. This patchwork of connections is prone to data packet loss and slow response times.
By focusing on building a strong network that removed the burden of building this from scratch and releasing simple, easy-to-use SDKs and APIs for developers, creators, and influencers, Tony Zhao and a few friends met at his house in Silicon Valley and wrote the first set of code to create the first prototype for SD-RTN.
With the rapid industry adoption of SD-RTN, Agora incorporated and opened its first headquarters office in Santa Clara, California. Today, Agora has grown to include more than 1100 employees with offices all over the world.
The Concept of the Metaverse
80.lv: During one of the GDC talks you discussed metaverses. What are metaverses in your opinion? How would you describe your vision?
Hermes Frangoudis: There is only one metaverse and it is a digital representation of our physical lives. The metaverse is an extension and rebranding of virtual worlds that have been around for some time and as a society, we adopted digital life some time ago.
When people hear the word "metaverse", they think of the many companies working to create their own virtual worlds that allow people to connect, socialize, transact (do business), and live as digital beings. But much of this is still nascent and it will take time to transition fully into 3D immersive worlds.
The transition from 2D virtual spaces on a screen to 3D worlds is not the only thing that needs to happen. The first fundamental change has already started but it was subtle. Ever seen someone standing there staring blankly into their phone? That person is in the metaverse, interacting digitally with content or another person.
Every day, society interacts digitally using existing social (metaverse) platforms, we can see it all around us. Slowly "all around us" will take on a whole new meaning as technology further blurs the lines between physical and virtual.
Developing the Metaverse
80.lv: How can your team help develop the metaverse? What are your main strengths?
Hermes Frangoudis: Agora enables developers to build apps and virtual spaces that support life-like interactive experiences where there is minimal delay between users as they interact digitally. Agora’s algorithms monitor the network in real-time and choose the most efficient routing path automatically, delivering an ultra-low latency of 400 ms or less. This is critical for a life-like metaverse experience.
Besides, Agora supports the most popular developer platforms for building metaverse experiences, including Unity, Unreal, and Cocos. We made it super easy for developers to integrate our APIs into their metaverse applications.
Agora also offers a wide range of features and complementary products that are critical for building fun and engaging metaverse experiences, including spatial audio, voice and video filters, or adding specialized features like noise cancellation and much more through the Agora extensions marketplace.
In-Game Real-Time Voice and Video
80.lv: Let’s discuss embedding real-time voice and video into games. How does the tech work? How can it boost the experience for players and help developers?
Hermes Frangoudis: When we talk about in-game real-time voice and video, there are a few parts: capture (collecting the data from mic or camera inputs), transmission (sending and receiving the voice/video data streams), and render (playback of the audio/video streams). Voice and video capture and render are relatively simple, the low latency connections in the transmission phase are the magic that makes the metaverse feel real.
Our fully customizable SDKs, pre-integrated third-party extensions, and low-code and no-code tools allow developers to quickly and easily build engaging social gaming experiences that connect and captivate users.
Our networking innovations handle routing in real-time, providing extremely low latency faster than other solutions, and delivering more responsive in-game communication. The initial video rendering is immediate, and our SDKs and network support adaptive video resolution and seamless channel switching to optimize a player’s experience even under the most challenging network conditions.
Agora’s intelligent network architecture automatically scales to meet global audience demand, with coverage for over 200 countries and regions. Our industry-leading 99.99% uptime/400ms average global latency provides the reliability and speed game developers need to meet their customer demands.
Integration to Game Engines
80.lv: How difficult is it to ingrate your solutions into game engines like Unreal and Unity? What are the steps?
Hermes Frangoudis: Agora’s Unity and Unreal SDKs are simple to integrate. Our Unity plugin is available through the Unity Asset Store, while Unreal is hosted on our Agora Developer Community GitHub.
Once you download the plugins, there are guides for getting started, but at a minimum, it is as easy as: import the library, create an instance of the Agora engine, join a channel, and pick which game element to render the video stream to.
Roadmap
80.lv: What is your current roadmap? What projects are you working on? Please tell us about your plans for 2022.
Hermes Frangoudis: Agora has a core belief that digital real-time engagement will be ubiquitous in daily life. What we see now is just the inception and we are barely scratching the surface of what is possible when virtual interactions are powered by real-time technologies.
From the workplace to healthcare to gaming, all the sectors are witnessing unprecedented digital growth. Agora will continue to focus on key verticals where real-time engagement is a significant factor such as gaming, online education, virtual events, eCommerce, and social platforms.
The focus is to create the tools need for developers to create the real-time enabled virtual world so as we move towards a post-pandemic future, we still see much of the convenience we have grown to expect for bridging the gap between physical and digital connections.