NVIDIA's Neil Trevett on His Vision of the Future of Metaverse

Introduction

80.lv: Please introduce yourself to our readers.

Neil Trevett, Vice President of Developer Ecosystems at NVIDIA: Hi, my name is Neil. My day’s job is at NVIDIA, doing Developer Ecosystems stuff, helping developers use GPUs. I’ve been president of the Khronos Group longer than I’d like to admit, since 2000 or so. But of course, the Metaverse Standards Forum is much newer. I’ve currently just been elected as the president of the Forum, or at least until the next election.

The Metaverse Standards Forum is very young, it’s less than a year old. We launched it back in June last year. And of course, the first question everyone asks, once they hear about that, we’re part of the Metaverse Standards Forum: “Finally, you’re standardizing it, you must know what metaverse is”. And of course, there’s no clear answer to what Metaverse is.

It’s gonna be a very Darwinian bottom-up process of the upcoming years to figure out what the Metaverse is going to be. But I think the reason why there’s a lot of industry interest and attention being paid to the Metaverse is based in reality because, although we don’t know the direction where it’s going to go, we do know today already is we’re bringing together multiple disruptive technologies, things like AI and Machine Learning which is in the press a lot right now, the GPU advances in rendering and computes in simulation, the whole XR domain, augmented and virtual reality, and Web3, although it’s a much younger technology and going through some growing pains maybe, there’s a kernel of a good idea there which is that we need distributed trust in databases, there aren’t under control of anyone or any organization. As we begin to mix these, it’s going to be disruptive in a number or different markets. So, I think companies are right paying attention so that they’re disruptors, not disrupted.

The second part behind the vision of the Metaverse which a real opportunity is to break house of the individual silos that we have today, like individual games or enterprise platforms, that’re doing in many cases very awesome stuff, and you can see the beginnings of the Metaverse in the games like Roblox and Fortnite. You can see it’s interesting to people, and in enterprise space there’re many companies deploying digital twins. NVIDIA has its own Omniverse, real things are happening. But they’re all individual silos, and if we can build the bridges between them, then we can begin to scale this new platform beyond those individual applications and then Network law kicks in which is the more people are connected to the network, the more valuable that network becomes. And potentially, the economy of the Metaverse can break out and be quite significant.

And the last thing, I think most of the people agree on is if the Metaverse is to reach its full potential, then it needs to be based on a platform that is open and inclusive to all, just like the web was an open platform that anyone could develop for, the Metaverse need to be the same thing. And these are the three realistic motivators why people are interested. The interesting thing is that interoperability is key to every single one of those. Getting multiple technologies to work together needs interoperability. Building bridges between silos needs interoperability and having an open platform meets open standards, and open standards are the way to get interoperability available to everyone in a pervasive way. Because there’re so many technologies coming together, the Metaverse is going to need a constellation of open standards, and it’s going to involve dozens, maybe hundreds of different standards organizations, each having their own specific area of expertise, and there’re many organizations doing great work out there. Interoperability isn’t a “nice to have” for the Metaverse, it’s the fundamental building of the Metaverse.

The actual timeline on how the Forum came about is the Khronos Group. We’ve been developing standards to the relevant parts of the Metaverse. We have Token3D, we have OpenXR, we have glTF. All these things are relevant to the Metaverse. And then we had suddenly a lot of people coming to our doors saying that they were really interested in putting standards for the Metaverse, but they were confused about the larger picture, so where do they go to talk about the Metaverse as a whole? And actually, we wanted the same thing. We wanted a place to go and begin coordinating with. The other standard organizations do adjacent work to us in the Metaverse. And we looked long and hard, but we discovered that there wasn’t anywhere for the industry to meet about coordination around the Metaverse standardization. And now, thanks to the Khronos Board, Khronos took the decision to see whether there was interest in the industry to create a Forum about Metaverse standardization, so Khronos has been funding a loose affiliation of companies that were interested to come together.

We decided to make the participation agreement quite lightweight. There has been a series of very straightforward agreements with the Khronos Group, and we wanted to do it that way because we wanted to get talking quickly, time was of the essence, we didn’t want to spend six months creating bylaws and incorporating, and then just discovering there was no interest in the first place. So, we put together this bootstrap phase, a simple agreement with Khronos funding, and we launched the Forum in June last year, we started with 37 founding members and by the end of last year we were pleasantly surprised by the amount of interest and we’re currently up to 2400 member organizations, and we have multiple working groups already up and running with projects to foster interoperability in the Metaverse.

I honestly thought we were gonna get a couple dozen more organizations once we launched in June, but we’ve ended up with over 2000. So, it was a good indication that there is industry interest. I think it does show that there are many in the industry that recognize that the Metaverse to reach its potential needs open standards and there’s willingness to get involved and an interest to get involved in the standardization process.

Interoperability in Metaverse

80.lv: On one side, there should be interoperability, and everything should work together with everything, and at the same time needs to be centralized. How do you make it work?

Neil Trevett: This is a good question and there are many different angles on that question to talk about them. It’s interesting, as you say, whether it’s Khronos or any other standards organization, W3C or ISO, no one has the authority to force anyone to use a standard. It must be and always will be voluntary, it’s really up to the standards community to make sure they’re creating standards that provide value to the industry, so we don’t force anyone to use any because we think it’s gonna be better, what we do, we try to identify the use cases and requirements and say: “We see that there’re companies spending money or wasting time and we think interoperability can help.” And with participation of the industry, we build standards and then offer it to the industry. And successful standards will add clear value and will save time, build markets, save money and therefore the companies will willingly and voluntarily adopt those standards.

That’s the way the adoption works. And there will always be a spectrum, on one end of the spectrum there will be companies whose business model is not based on using standards, and there’re many companies that are actually anti-standards, they want to have a walled garden and keep people locked on their platform. And that’s perfectly valid. It’s not a criticism, some of the most successful companies do that. And on the other end of the spectrum there’re companies that definitely believe in networking and multiplicative effect of using open standards and not having to do everything yourself, building a larger network. And there’re companies all up and down that spectrum.

Again, I think it’s the role of the standards community. We can’t dictate business models to any company, but we should enable the companies who want to go towards the more open end of that spectrum. We should have standards in place that give them the choice to go to that end and enjoy potential benefits if they choose to. And then you mentioned decentralized, it’s interesting, I think it’s the Web3 thing, and Web3 is kinda Wild West, but as that technology matures, again I think there’s a really central idea that if you can have a centralized trusted ledger that is decentralized, it is gonna be blockchain? I’m not an expert, I don’t know. I mean, there’s lots of people of course researching and figuring out the best way to achieve that decentralization. But even if people can use the ledger however they want, you still need interoperability standards cause everyone needs to use ledger in the same way.

Choosing the Unifying File Format

80.lv: Currently, USD is slowly becoming the standard format for 3D programs, uniting all the versatile 3D software into one. What's your take on what the unifying system behind Metaverse is going to be?

Neil Trevett: Yeah, that’s a great question, and something that’s near and dear to my heart. I think the industry is beginning to narrow down the number of 3D formats into some good candidates, and it’s an interesting and undecided debate whether we gonna end up with one or a small number.

Where we’re right now is both USD and glTF are awesome, and full disclosure, as you probably know, the Khronos runs glTF. USD is actually interesting, and it’s not a criticism either, USD is not an open standard, it’s an open-source project. There’s no USD specification but maybe for what it’s trying to do, it's the best thing, so again it’s not a criticism. It comes from Pixar and now it’s being used by Apple, NVIDIA, and many other companies, and has an amazing scene composition and offers such capabilities that you can have a team of a hundred designers working in one USD database.

And then there’s the glTF that’s coming from the Khronos side, which is absolutely coming from the opposite direction. We need to be able to have something that’s as deployable as JPEG but for 3D, we need to be able to make sure that if we send a 3D asset to an 8-year-old Android phone, it’s gonna run, that’s fine because glTF has the Web. Key characteristic of the Web that is has to run anywhere, including on my mom’s 8-year-old Android phone.

So, both completely valid, interesting and enabling use cases, but they’re coming from very different directions. Now, the interesting thing is that of course they’re gonna meet in the middle at some point, in fact, they’re already beginning to meet in the middle. Now, there’s USDZ which is the Apple subset, which they’re using for a real-time deployment and a lot of folks who use USD say: “Okay, I’m offered all this wonderful stuff, how do I get it out there?”. And the glTF is going: “Okay, we’re really good at describing a single asset, but what happens if I want to have two glTF assets in a scene? I need a way to start composing simple glTF scenes without suddenly need all the open big project which is USD”. For composing a couple of chairs in a room, you don’t need all the complexity of making a movie. But they’re both naturally evolving and they’re beginning to meet. glTF is beginning to think about composition, USD is thinking to think about runtime.

It’s actually one of the first working groups of the Forum. We have a choice as an industry, are we just gonna let the two crash into each other and wait, picking up the pieces once the fight is over? Wouldn’t it actually be better if we realize what’s going on and begin to talk to each other before it becomes a crisis?

And in fact, it’s exactly what’s happening at the Forum. One of the first working groups at the Forum was USD/glTF interoperability working group. And it’s actually really interesting. It turns out, of course, many of the folks on both ecosystems, they already know each other. The 3D community isn’t that large but because they haven’t had the opportunity to talk with a specific goal of getting USD and glTF to complement each other, and because no one has set out the table with the chairs saying “Here, peace, we gonna sit here and talk about this”. And it turns out that everyone has been wanting to talk, everyone is actually quite motivated, the glTF guys don’t want to reinvent USD, and USD saying: “Actually, we want to deploy in the Web, and glTF is actually an awesome way to get assets in leaf nodes of our composition tree”.

But we’re discovering that glTF doesn’t have quite enough animation flexibility to be round tripped through a USD offering sweep. We’re finding that USD really wants to adopt the PBR model from glTF which has been battle-proven across many different devices and platforms. So, it makes me happy to see this working group, all these folks, it turns out, they have good intentions, and through working together we can really benefit both ecosystems and we will see how it goes in the long-term. But in the short-term there’re definitely things that we can do, for example in the 3D commerce meeting at Khronos this morning we were talking exactly about this topic and said how awesome it would be to import my glTF, use my USD tools, and we export and it’s not all screwed up. That would really help my business and that’s what we’re working on. We think that will be super important in the bigger picture in the Metaverse, but right now if we can help solve it, it will make people money and that’s the motivator for doing it again and again down the path.

Storing the Files

80.lv: Could you please tell us more about asset management and the protection of the creators' IPs? Where will Metaverse's files be stored? 

Neil Trevett: When we say interoperability, that’s a very important term, one of the first issues we’re going to have to figure out incrementally, not all at once, but incrementally we will figure it out is just because you can, if you solve the engineering problems that we were talking about just a minute ago, if we solve the engineering problems and you can take 3D from place to place, should you?

And there maybe many reasons you actually don’t want to. It’s just like in the real world, you can take physical objects from country to country, but there’re many examples of things you shouldn’t take from country to country for various reasons. In the virtual world, there’s gonna be IP licensing reasons why some companies, you know, an IP, a brand, Star Wars thing you bought over here, the business model might not be in place for you to take it to other places, so you need to have like a customs post where the ownership and the IP licensing and the age appropriateness, maybe even the styling, cause if you take assets from one game it may be very horrible and inappropriate if you take to another game; age appropriateness for safety, and all kinds of things.

So you can definitely see that just because we can take everything everywhere, in the real virtual world people are gonna want to set up these checkpoints but again you need interoperability because you want to set up a checkpoint that can process everyone’s assets, doesn’t matter where they’re coming from, all the fields around, you mentioned metadata, we need to have some interoperability. How do we store the metadata? How do you know what the IP license is? How do you know who the owner is? And this is where blockchain maybe comes in.

We need to put the building blocks together and then the next layer up in the stack will figure out whether we have the right building blocks and if we do, they can start building their layer and that’s why this cooperation between different types of standard organizations is going to be so important

I’ll use glTF again as an example, because I know it, because it’s from Khronos. We can build in a field in glTF, in fact we have already. It’s based on XMPLD, it’s an ISO standard for metadata. You can now have a metadata field in any glTF asset, that’s what we can do, we own glTF spec, we know how to do that, we can get that done. We’re not though the domain experts in the next level up, what are gonna store in that metadata field for the kinda more the Web3 type interactions: ownership, identity, security, and we need someone else, a different standards organization to worry about that stuff. We need to tell them to tell us that the glTF metadata field sucks because we needed three fields, not two, or whatever it is.

And we’re happy to fix that but we need to be communicating with the folks that are gonna be actually using the nuts and bolts that we can put together in glTF and wherever you go in the Metaverse, there’s gonna be that layering effect that’s gonna be standard for the lower level which normally gets solved sooner and once the solution is available, people begin to use it, evolve and innovate in their own layer but if the glTF group never finds out that their metadata field is broken in some way, it won’t get fixed and we waste time, and people above us are not able to innovate as quickly as they could, And again, that’s why the communication is so important since we can’t go into this complicated road map which we don’t know where it’s going.

Metaverse's Current Condition

80.lv: From your point of view, where is the Metaverse now? What do people think in general about Metaverse? Is it something like an idea that is still in its infancy or are people actually having plans?

Neil Trevett: The Metaverse is going through a bad patch of PR right now. If you look at the tech news, there’re multiple large tech companies that lay off their metaverse teams and a lot of analysis around Meta and other companies on what they should do and that things aren’t going well and last year it was the best thing in the world and this year it’s the worst thing in the world.

We’re gonna go through multiple lifecycles, there’s no doubt, we’re on a 20 to 30-year journey here, and some quarters it’s gonna be great, some quarter it’s gonna be awful. The way I look at it, it’s why I always start off the presentation not with some grand vision that we’re gonna be living in a virtual world. We’re just in the process of bringing all these interesting technologies together and that’s gonna create interesting opportunities, and that’s something I actually really believe in, no matter how long the Metaverse or whatever it takes, whichever direction it goes, it’s something that’s happening today and it’s clearly happening today.

The Metaverse as a concept has suffered from being overhyped so it’s natural that there’s gonna be a backlash, but the backlash gonna fade and everything’s a cycle, and then people will get excited by it again. One of the values that standardization community in any standards and hopefully the Forum, however long we last, one of the values as well as doing the heavy-lifting of getting things to work together is that we provide continuity because the Forum has a bunch of organizations, you know this standard organizations have hundreds of member companies, individual members within those consortiums, their quarterly results will say: “Okay, this year we’ll have to wait”. Individual companies will go in and out but overall, it will average out and there will be sustaining energy and momentum to provide continuity so things will continue to evolve, the players might change but the direction and the goals remain as long as you can maintain bigger enough quorum.

You can make forward progress and there always will be people who will leave, and they will come back and someone else leaves. It’s okay because the average energy remains, and you can make forward progress. I think the Metaverse is gonna be that kind of a cyclic thing so we’re gonna need that continuity. We shouldn’t believe the hype. Back in the heyday of the Metaverse, they were forecasting it’s gonna be a 2.5 trillion dollars opportunity in the next three years. Oh my God, what were you smoking?

And I think the Metaverse has some other PR problems which we will get worked out as we go round the washing machine cycle. XR is getting better every day, but the Metaverse is being over associated with XR, you don’t need to be plugged into an XR device to use the Metaverse, in fact if you look at the most proto-metaverses like Roblox, Omniverse, Fortnite, no one is using XR. The closest thing known to the Metaverse, and no one uses XR.

I’m not talking down, XR’s gonna be great but this notion that some of the media have said, if you’re not plugged in, isolated from your family in an XR device, you’re not in the Metaverse, no, that is not helpful or accurate. Actually, I think the other over association is, another portion of the world thinks that the Metaverse is NFTs. You know, Web3 is the only part of the Metaverse that matters and some of the bad behavior that’s going on over there is unfairly tainting the Metaverse. Yes, the decentralized ledger or whatever ends being is gonna be really important but it’s not NFTs.

People scamming each other with NFTs – I don’t think it’s a criticism of the Metaverse, it’s been over associated there. And actually, several design companies talked to me and said that they were really interested in the Metaverse, and they loved what we’re doing, but their clients can’t use the Metaverse because they don’t like the NFTs.

It’s a real problem. What the Forum can do along with everyone else is try to set the record straight and not get too pulled into the hype and not get too pulled into the doom and gloom Because we need to be the thread of continuity, pragmatic, eyes wide open but making sure that we take advantage of the opportunities that are coming because all these fantastic technologies are beginning to be combined in new ways. 

Neil Trevett, VP of Developer Ecosystems at NVIDIA and Chair at Metaverse Standards Forum

Interview conducted by Kirill Tokarev