The “Inside Out” of Pixar’s Renderman

At SIGGRAPH we met with Chris Ford, the Business Director of Pixar Animation Studios. He talked with us about the ins and outs of RenderMan (Pixar’s technology for rendering animation and visual effects), his favorite Pixar film, and some tips on getting hired at Pixar.

Chris Ford

My name is Chris Ford, I’m the Business Director for RenderMan. My first college degree was in graphic design and then I did computer science. I worked in production for a period of time, but I actually reached a point in my career where I realized that the means were actually more interesting, stimulating, and fulfilling than the ends. Most people are working in a creative role because they are creatives. What they produce is what gives them job satisfaction. For me, how to get there gave me the most satisfaction.

I was actually on the original team that developed Maya way back in the beginning. I was the Product Manager for Maya from version 1. I joined AutoDesk for a period of time and I was responsible for 3D products such as 3ds Max. Now I’ve been at Pixar for the past 10 years. The common thread for all of these is that I’ve become very interested in all the tools and how you do things. How you get around problems and solve them is the most fascinating part of this business. Also, seeing new methods of achieving a certain type of effect or result is definitely very stimulating.

RenderMan

RenderMan actually goes back to the origins of Pixar when in 1986 Steve Jobs purchased what was then the computer graphics division of Lucasfilm and turned it into a company called Pixar. Pixar was originally focused on selling hardware and software to process, what then in the 1980s was high-level computer graphics and obviously pretty primitive in today’s standards. It was only in the early 1990s that Pixar started working on the first fully CG animated feature film, which of course became Toy Story, and since then has become best known as a feature film animation studio and indeed, that’s what we really are.

However, that basic technology core in Pixar, especially in the area of rendering, has always remained and we’ve always shared that technology with the rest of the industry. In fact, when Toy Story was released, it was right around the same time another landmark movie was released which was Jurassic Park. That was the first movie which really had fully CG-rendered dinosaurs, but they were good enough to be combined with live-action in a way that was believable. Since then, RenderMan has essentially been behind much of the modern revolution in visual effects in producing photoreal imagery. So when you look at various landmark movies such as Titanic, The Lord of the Rings, Avatar, The Avengers, or various Pixar movies, all of them use RenderMan to generate the photoreal images you see. In fact, RenderMan has been responsible for 19 out of the last 21 winners of the Academy Award for Best Visual Effects.

That’s the part of Pixar that most people are not aware of, but is actually a fundamental part of what Pixar is. It’s both a very successful and creative studio that produces amazing animated feature films, but it’s also a studio that’s responsible for much of the key technologies that are used in the modern visual effects and animation industries.

Photo Real RenderMan

RenderMan is a photoreal renderer. Sometimes it’s nicknamed “PR RenderMan” for “Photo Real RenderMan” and our mission is simply to make images look great on a movie screen that’s 50 feet across. We’re talking about extreme levels of photorealism here. In a modern feature of animation, especially visual effects, you may have like 20, 30, or 50 layers of imagery all composited together: live-action, computer graphics, mattes, and other various components.

All Pixar movies have a certain look which like an abstracted realism. It’s the visual style and look of the movie. The Incredibles looked very comic book cartoony, Ratatouille had a french impressionistic painting look, Brave had a very green landscape, and Cars had a Southwestern landscape. But, RenderMan is a photoreal renderer and in visual effects photorealism is exactly what it’s about. So that’s our focus, photorealistic imagery for the movie theatre in such a way that the audience is totally fooled and they can’t tell what is real and what is not.

Achieving Photorealism

Actually within the past couple of years there’s been a mini revolution in rendering. Traditionally, rendering used what is known as scanline-based techniques. Technically, these techniques were sometimes called biased methods. To put a fine word on it, it’s kind of a cheat [laughs]. We kind of rendered photorealistic scenes by making certain assumptions and basing it essentially on how the camera sees the scene, and calculating by each scanline what it is the camera would see for each pixel.

Over the past few years we’ve seen a move to physically based methods of rendering using ray tracing. Now these are not new, ray tracing has been around as a technique in computer graphics for many decades. However, it’s only been over the past couple of years that the price of computing has reduced to such a point that physically based ray tracing has become a viable method, and this is a technique which is often referred to as unbiased rendering. So, we’re actually physically simulating how light travels from a light source back to various surfaces or refracts through various substances like glass or water, eventually traveling through the image plane to the camera. It’s as real as we can make it, because essentially we’re following the physical characteristics of surfaces and substances that light travels through. Admittedly, 100% is never possible because there’s always that last few percent. But even so, we are actually simulating physics now and that’s a big change in rendering which has just recurred over the past few years especially in feature film rendering.

RenderMan is available as an add-on for Maya but is this product available separately and do you have plans to build this product in a separate category of tools that will be available for the public. RenderMan is a standalone solution. We have various bridges or plugins to authoring applications. Maya is certainly the most common, but there are others we have here at SIGGRAPH. We’re showing a Blender to RenderMan connection. We are working on an upgraded Houdini and Cinema 4D bridge to RenderMan. We can take the output of many different types of authoring applications. We have many users, especially larger studios who often have their own pipelines and they write their own data in different ways or they want to build the renderer into their own pipelines.

RenderMan has always been very flexible. It’s always had the ability to be customized or to be embedded into other applications. From a normal artist perspective, yes, you would use it in Maya or another application and it would be a very Maya-centric workflow. You just pull down menus and images would pop up. However, it can exist as a standalone application as well if you have your own data that you want to channel into it.

Challenges 3D Animators Face

I think you gauge the health of an area of technology by the amount of research and the papers published in an event like SIGGRAPH. Just a quick glance that highlights that, this is a very active area of research. If this was an area which was very mature, you wouldn’t see that same energy or same vibrancy of research going on. There are many challenges to be solved. I think that’s the immediate takeaway.

Let’s talk about film rendering. We’re always asked if we can produce human beings that look so real that you can’t distinguish them from reality. From the rendering side that’s pretty much resolved. There are great still images of people who are hard to tell apart from a photograph. The challenge there has more to do with emotion and other aspects that have to do with producing synthetic characters.

There are other challenges in rendering as well like volumes and how light passes through mixing volumes of different substances. That presents a whole category of issues that are still to be addressed. There is also performance and speed, for a large cinematic image on a movie screen. Some images can still take 9 or 11 hours each. In fact, there are horror stories of hundreds of hours per frame in some really complex shots. The physically based methods, especially, that I referred to earlier still are very time consuming and take up a lot of compute power. On the other hand, we have new technologies and options like the cloud and the ever-increasing number of cores and processor architectures that continue to speed up certain things.

In rendering there’s a well-known rule called Blinn’s Law, named after Jim Blinn who is one of the pioneers of computer graphics. The law says that a rendered frame will always take a certain amount of time no matter how fast the computer or how efficient the algorithm. Blinn’s Law has always been true, and I suspect it will continue to be.

Definitely you can always see people producing great looking imagery in real-time on GPUs. You see this with NVIDIA and game engines, but they tend to be a more limited subset of visual graphic rendered imagery. But for feature films, we’re dealing with the entire totality of the physical space as lit in an outdoor and interior environment with all the visual effects and everything else that’s going on. That’s still something that’s not yet possible to do on GPU. It generally tends to be a CPU based process on very large render farms with many cores rendering away. There are many challenges in this business but those are maybe just a few [laughs].

Size of Development Team

The development team behind RenderMan is actually pretty extensive. First of all, we’re part of Disney and they have several research groups that focus on computer graphics in all areas. They have research teams at Carnegie Mellon University, Zurich. Pixar has its own research group and Disney Animation has its own research group. RenderMan is actually a conduit through which some of that technology can be channeled into the broader production industry, especially where we want to encourage the use of standards so that different studios can exchange information.

If you look at any movie, you see all these studios at the end working on different shots, but they’re all working with the same assets so we need a way of sharing all those. So we have a really large development effort which channels the whole Disney group into RenderMan. The RenderMan team itself is fairly large. We’re obviously part of Pixar so we service the studio and we also service the whole production industry. Our research base, I would say is by far the largest in the industry and we’re focused very much as a set on cinematic rendered imagery, which really is at the extreme level of fidelity. If it’s not good on a big screen, it’s not going to work on any other format.

The Confidence of RenderMan

The key to it is that we have built a production community of users that are good at identifying challenges. The question we’re often asked is why does Pixar make this technology available for the industry to use? The answer is that it’s very much in our interest to have the software stress tested and tortured in various different applications by many different projects, because that then feeds back into the software and makes it very robust. 50% of RenderMan, I think, is the confidence that it can deliver a movie with a $200 million budget on time. That’s really what our reputation hangs off of.

We balance our development and the effort we put into the core stability of the software very carefully. This is a fast moving technology so we have a significant portion of the team that’s focused on innovating and bringing up new functionality and new features every year. Generally, at the moment, we’re aiming at two major releases of this technology each year, just as a testimony to how fast this industry is moving.

Accessibility of RenderMan

Once a new version of RenderMan is released, it’s released to the world at once. Pixar doesn’t have a special version that we keep to ourselves or that we use early. I mean, there are some things we try out in beta that we try out in Pixar or some technology that the studio has developed that we channel outwards in the industry.

In fact, we’re talking about that here at SIGGRAPH. We’re making available the hair shader which you last saw on Sully in Monster’s University as a technique called Marschner Hair. We’re making that available for the whole entire industry to use or anyone to use in the current release of RenderMan.

RenderMan is available for free for non-commercial use. Anyone can go to the Pixar website and download a copy for free. There’s no time bomb, watermark, or limitation. It’s fully functional. The only limitation is you can’t make money with it, but if you want to experiment with it and use it that’s fine.

We did that deliberately to channel into the industry the type of technology that Pixar and Disney is working on, in the cause of encouraging open-access, open standards, and a foundation upon which people can build tools and plugins in the future. This comes back to what I previously said. RenderMan is very much part of the soul and character of Pixar as a studio and who we originally were which was a technology innovator as much as a studio, which is now what we’re known best for.

Targeted Audience

Our development focus is driven by cinematic imagery, that’s who we aim for. For example, we don’t target ourselves specifically as a games renderer, a visualization renderer for architecture or for cars or for product design. Our focus is primarily on feature film rendering. That’s because that level of visual quality and fidelity trickles down and that’s what everyone aspires to.

This is an industry where no matter what area of the visual media you’re working in people will generally say that they saw that in a movie. That’s what I want. That generally tends to drive aspiration, but that doesn’t mean RenderMan isn’t applicable to those applications. RenderMan’s been used in games especially for FMV and cinematic sequences, it’s been used in scientific and data visualization, many of the NASA type high-level imagery and animations you see use RenderMan in some form. It’s been used in product design and visualization, and definitely a lot in broadcast and television, but film is what sets the bar in terms of development because I feel that’s where the ultimate stress testing is.

Games that have used RenderMan

Games have used RenderMan for their cinematic sequences. We’re not an engine so we don’t compete with Unreal or those type of solutions. Our area is on high-fidelity visualization for movies that aren’t in real-time.

Artist Friendliness

The way that technology has evolved, has definitely been to be artist friendly. We see this trend not only in movie visual effects and animation, but in other areas as well. Traditionally, rendering was sometimes thought of a bit as a black art, you needed to be a programmer or have certain skills to do it. All the trends are moving away to artist friendly workflows. So, ease of use and artist friendliness is absolutely a key development priority for us and it’s really where this industry is going.

Getting Hired at Pixar

First of all, think of Pixar as a left brained/right brained studio. We have intuitive thinkers, we have logical thinkers, artists, and engineers. Pixar has a lot of people who actually come from both areas. We have artists who feel quite comfortable using software tools. We have engineers and developers who actually have pretty good artistic skills. Being open to both sides is something Pixar traditionally has been strong at. That’s typical for computer graphics which is a great mixture of art and science. That just really sums up Pixar. We are both art and science.

We look for those type of skills and when people apply to our studio, in our recruiting process we look at the work they have done, individual creativity, and standout innovative thinking. We don’t necessarily look for people who have skills in any particular software or anything like that, we can always teach that. We have an internal Pixar University which is devoted to teaching skills to people internally. What we really need is the flexibility and mental agility to be able to learn new things and to apply the skills you’ve learned in other applications to our own pipeline. We are always posting open positions on the Pixar website.

Favorite Pixar Movie

For me it would have to be The Incredibles and that’s because I’m a big comic book fan (I especially love Marvel), everyone has a different answer to that question [laughs]. But actually there are different movies that are different milestones, and actually there are milestones in rendering that you can associate with certain Pixar movies. If you look at the original Toy Story, everything is kind of plastic because that’s all we could do at that time. You know, everyone was just smooth, shiny, plastic surfaces. There wasn’t any hair, fur, or anything complicated.

A few years later Monsters Inc came out and we had a few, well, one furry creature because that’s all we could do. And by the time we got to Ratatouille we had lots of furry creatures. Nemo introduced underwater caustic lighting effects, which were very distinctive at the time. All of those were different techniques that first appeared with Pixar movies.

There are definitely movies that had certain technical challenges that we are particularly proud of overcoming. At a personal level, all Pixar movies are about the story and if the story is right, it doesn’t matter how good it looks, the story is everything and you can put up with a lot if the story is right.

Chris Ford, Business Director, Pixar Animation Studios