We’ve decided to prepare a report on an amazing presentation by Anastasia Opara that was held at the GDC 2018.
Take a survey
We’ve decided to prepare a report on an amazing Houdini presentation by Anastasia Opara that was held at the GDC 2018. The talk was mostly focused on how to approach the procedural layout generation and placement on the example of the PICA PICA project that was put together at EA’s SEED. SEED is a research division of Electronic Arts that contains a cross-disciplinary team within EA worldwide studios. SEED’s mission is to explore, build and help define the future of interactive entertainment via creative and technical exploration.
One of SEED’s recent projects is PICA PICA. Project PICA PICA is an exploration of hybrid real-time raytracing rendering, deep learning agents, and procedural level generation. The result of the project is an exploratory minigame for AI agents built-in SEED’s Halcyon R&D engine, where AI learns how to navigate and interact with the environment via reinforcement learning.
Art team consisted of three people in total. In the end, for time efficiency team decided to have most assets as pre-made to devise a workload, though some parts such as walls, floors and conveyor belts are still procedurally generated meshes. At some point, the team had to have almost a full restart of the art, in particular as a procedural system, 90% of which was recreated.
The study brought up a few important questions such as what is appealing, what is boring and how to define it procedurally? Rules are often thought as something rigid and constraining when it comes to an art process and an idea of defining your art direction in a set of very specific instructions might seem like a daunting, if not an impossible task. A lot of times a creative process is a fascinating dynamically evolving collaboration and contribution of both conscious and non-conscious decisions and the latter are very hard to trace. Why did we decide that one color is better than the other or that particular arrangement feels pleasing and relaxing? Even though we do not directly register it, we are pattern-seeking addicts that are always in a search of the next fix, that will allow us to better encapsulate our surroundings into inner representations and better means more compact but not necessarily more truthful. Our brain lies to us a lot. We generalize, assume, use stereotypical thinking, mix opinions with facts, but these are actually amazing mechanics that allow us to process a huge amount of information on a daily basis. Just imagine you had to learn a concept of a car or a house every time you face a new data point. You wouldn’t be able to leave your house in the morning just because you would have to be familiarized with your own bedroom every time.
The main question is what makes us label things as appealing or boring? It might seem like a very trivial question but you’d be surprised how automatically this evaluation happens in our brain and how often you do find yourself feeling that something is off. It takes a considerable effort to translate it into words or feedback. If we do define what’s appealing and boring and reverse-engineer what tickles our brains, we can use this knowledge when it comes to defining our own procedural rules set.
A visual example can be expressed as a pentagon. It is a simple shape with easily deconstructible rules and our brain effortlessly traces the dependencies. Since this is too simple and familiar it can be described as boring because it doesn’t introduce any new information for our brain to prove its existing models, and that is why it is a very predictable model.
Adding just one more simple rule on top of it such as copying every point of the pentagon n-times with a certain scale and rotation and repeating the simple rule again, we get a more interesting looking shape. All of a sudden, the brain is engaged because it is challenged with the higher dependency. It is told to make sense of the raw visual input, as there is some kind of underlying rule set and configuration similar to the ones we engaged before but with a slightly different twist. So the brain starts to search for a pattern once it predicts it is possible to find one. This process of revealing previously unknown dependencies gives us pleasure and may have roots in our primal survival reward mechanisms. On the other hand, some dependencies can be untraceable. Noise, for example, is quite tiring as our brain simply can not find any pattern to cling to, it’s perceived as incomprehensible and unrelatable to any previous data and therefore it is also labeled as boring or unengaging. So the maximum of the interest’s function has to be somewhere in the field of our existing models of the subject but with variables, your brain finds engaging enough.
But defining and finding an interesting-looking procedural object is only the first step. Scaling generation up by introducing many instances of it is a different task. House generation system that creates some sort of a village suddenly brings new challenges. Despite having an object with the stimulating rules, a completely new situation is established, where brain no longer looks for those details in a single occurrence but analyzes everything in a big batch at once.
A computer will see that generated village as a set of different houses, but for the brain, it is easy to establish that those are copied objects with slight permutations. Facing little to no challenge brain becomes bored. Surprisingly by increasing the number of object occurrences, its overall interestingness is decreased. Therefore a good point to keep in mind is supplying engaging dependencies on every scale and rules are just building blocks in a highly creative process of arranging them into new patterns and dependencies.
Taking the same pentagons’ system but with various input shapes that produce big differences from the same simple setup. This is important for setup to maintain an engaging structure because it is the exact problem team faced in the first iteration of their level layout system. It was way too unstructured and random, there was no engaging internal logic. It is crucial to supply logical constraints that are tied to existing experiences. In this case, it was an idea of a factory.
In order to counter that, the system was redesigned to have a central piece and compositional guidelines. Once those blocks were in place it was easy to start building on top of that.
Rules assembly can be seen as a language, and language is one of the most amazing inventions that is aimed to align our inner highly varied data representations to a more generic and translatable, between multiple parties, form. That form is not necessarily the conventional communication, but literally, anything that can be used to trigger your internal abstractions and pass some sort of information. Language can be visualized as a handle that triggers certain inner connections.
Reading the word “rabbit” points to a certain location of our inner data that is also intertwined with a lot of other associations both cultural and biological that may be related to the concept of the rabbit. The biological meaning may be described as harmless due to our ancestral heritage of interacting with species. Cultural meaning can be an antonym to a wolf as those are usually put together in fairytales as opposites or a symbolic meaning of innocence. What is more interesting though is combinations with the word and how you can manipulate an association chain. Poetry unravels new words connections that can twist brain into bizarre associations. It raises the question about the nature of creativity. From that perspective, it can be described as a process of revealing previously unfamiliar connections and combining concepts into new mixtures and thus creating new meanings. This may be the reason why procedural generation is such an addicting activity of discovering models that describe the visual surrounding world.
Language can be presented as a constructor or a system of words that are linked together to pass information. Some of the word functions require previous data to be interpreted correctly. For example, two sentences, Sam jumps over a puddle, and he landed right into it, lead to two logical links. The first is that he is Sam and the second link is it is a puddle. But without the previous sentences, we wouldn’t be able to derive links from them. Therefore there is an informational link between those words and those links can be very complicated depending on the text. You have to build vocabulary to be able to express yourself better with this language and then just play with words. Digital assets are new vocabulary.
Sometimes the sentence just comes to mind and all it takes is to write it down, but often this is a process of playing around to get to know your idea, flirt with it, learn how to better approach it, be experimental, mix things up in a non-obvious way. The creative process is often very messy, and that’s alright. The process goes through many iterations of the algorithm before it becomes the amazing link between words or rules that just fit beautifully together. And when it comes to building the rules, it’s better not to try to do everything at once. The right way is to start simple, treat proceduralism as if it was a painting where you begin with bigger shapes, the foundation and proportions and then go into smaller details. That is actually a point that played a very important part in the creation of a procedural system for PICA PICA which kick-started an interesting research on visual balance and composition. The goal of the research is to understand how to create a visually pleasing layout and assets arrangement.
It is worth noting that the key idea behind the work is based on the book called ‘Art and Visual Perception’ by Rudolf Arnheim. Anastasia took the compositional lines and started sketching level layouts and how she would manually arrange them on a canvas and recording what exactly was happening in her brain. There was a very curious regularity here. The artist noticed that every time she created misbalance at first based on some intersecting compositional lines and then she had to balance it, relative to the center of the canvas. For example, imagine you have a plain surface of a size of your canvas that is balancing on the tip of a triangle which is perfectly stable until you put something on it. To deal with this mass, you have to put something else in a very specific location with a certain mass to try to bring this imaginary scales back into the equilibrium.
The composition is a huge topic that is based on shape, value, direction, size, rhythm, and other parameters. Instead of trying to encapsulate all of them, you should deal with the size as a visual parameter of the mass function of an object. So how does one do that? Think about Newton’s law of universal gravitation as it all comes down to compositional gravity. You can forget about constants for now – just create your own ones.
Anastasia managed to mix both Newton’s formula and her modified compositional gravitation formula in order to see how different forces affect her canvas. In the end, you can modify the strength influence distance and the spread of this force, but that’s just one of the things affecting your canvas.
Let’s imagine you have some initial disbalance. How do you deal with it? Where should you place a counterbalance? You need candidates which you should sample randomly based on normal distribution and their direction. Just remember that your goal here is to balance out the canvas so that the center goes back to equilibrium (zero vector with all the forces cancel each other out). The main thing here is that the magnitude should be as close to zero as possible. The artist showed how you can do that closer to the end of the video, so make to check out the needed math in order to achieve balance.
There’s actually a simpler way. You can just randomly sample masses to find a good, so you don’t need an algorithm to be too perfect since you’ll be adding other elements later. Imperfection and a chance of randomness are not bad in this case. Don’t forget about it.