Daniel Santalla explained how he set up the mechanic that allows the player to draw on a blackboard in VR and how it can be done with the glass surface similar to the effect in Half-Life: Alyx.
My name is Daniel Santalla, I live in Merida, Mexico, and I work at Recreate Games as a Technical/VFX artist. I started my game dev journey when I was 16 as a 2D artist but have jumped between roles ever since, going through game design, audio, and programming to finally end up as a tech artist doing shaders and visual effects. I studied interactive design at the Universidad Modelo and at the same time I co-founded Cometa Games, where I developed mobile games like Pingo Park, Beast High, and Monkeynauts.
Today, I'm working on Party Animals as a tech artist with a lot of very talented people. Also, I usually create educational VFX content that I share on my Twitter page.
Thoughts on Unity
I think that the best thing about Unity for a tech artist like me is that it is a really hands-on engine. You have to build most of the systems yourself and this is the perfect way to spark creativity and create unique concepts.
Idea of the Project
I actually came up with the drawing mechanic after creating a similar effect that highlights a gameobject. All I’m doing here is using a distance node that calculates the distance between the gameobject’s position and the fragment’s world position.
This effect sparked the idea of having a shader that could read past positions of the object to simulate a trail. The obstacle here is that the shader graph doesn’t support vector arrays (yet) so I had to come up with a more elegant solution.
The way I addressed the mechanic is a bit hacky but is really easy to pull off, even when you don’t have a lot of knowledge about shaders.
The setup is using two key components: an orthographic camera and a render texture.
At this point, you’ll be able to draw on the board anything you want. Using culling masks you can create a layer for your painter objects and create things like tips for markers, stamps, the tip of a pencil, etc. Here you can check the system working with a brush in a game I’m making called Art Studio VR:
Recreating a Similar Effect from Half-Life: Alyx
In Half-Life: Alyx, a noticeable part of the effect is that you’re not drawing on a blackboard, but instead you are painting over a transparent window. This is actually a really simple thing to implement into our workflow. All we have to do is replace the default Material with a custom shader that supports alpha tweaking and transparent rendering.
Pen Pressure Simulation
One thing you’ll notice when testing the mechanic is that you’ll naturally lean into the board and will lose immersion when you don’t feel physical push back on the pencil. This leads to an unnatural feeling of not being able to control the pen pressure. A way I’ve discovered to cheat this sensation is to use different tip shapes and rely on the natural motion of the hand instead of actually detecting the force pushing into the board.
Is Color Mixing Possible?
With this workflow, mixing colors is simply not possible, because the camera renders new frames on top of the others and there is no straightforward way to mix between them.
If you have any questions or you came up with anything cool involving this technique, my Twitter DMs are open, please let me know!
To learn more about tech art, check my Twitter page.