LogInDivesh Naidoo spoke to us about the story and development of Simulon, a cool application for adding 3D models to real-life footage, discussing the tool's features and explaining how Jurassic Park played an integral role in its creation.
In case you missed it
You may find these articles interesting
Introduction
Hi, I'm Divesh. I studied at The Animation School in Cape Town. The experience was amazing, however, most of my skills were acquired through self-learning and trial and error.
My journey took an unconventional turn when I chose to leave college early and build my first company. My first entrepreneurial venture, Dondoo Studios, focused on creating VR/AR training simulations for the industrial sector, serving multinational clients like Glencore and Anglo American. With Dondoo, I gained experience in building physics simulations, working with complex custom hardware systems, and creating software based on real-world 3D data.
I've been lucky enough to have worked on many interesting projects over the years that included the creation of custom VR camera and drone rigs, interactive projection mapping systems, large-scale 3D reconstruction, virtual production, and more. Additionally, my experience working on VFX productions in the South African commercial industry has contributed to my understanding of practical VFX pipelines.
The History of Simulon
Simulon started its journey in 2019 and has grown to a team of seven highly dedicated and talented individuals. Each member brings a unique blend of skills, passion, and creativity that is the driving force behind Simulon's success.
A notable aspect of our journey was during the COVID-19 pandemic when we chose to live and work together in a scenic South African location. This decision not only allowed us to maintain our productivity but also fostered a deeper team bond at a time when external circumstances challenged conventional work dynamics.
As the world transitioned back to normalcy, we adapted to a fully remote working model, which has proven to be exceptionally effective for us. This flexibility allows each team member to contribute their best, irrespective of their location.
We thrive on a blend of ongoing research, rapid testing, iterative development, and foundational implementation. Our tech stack is designed to be modular for swift and seamless improvements. As for joining our team, a solid understanding of 3D software, real-time engines, full-stack development, and knowledge of artistic communities would be ideal. But most importantly, we value self-driven individuals with a history of engaging in innovative projects. If you would like to be considered for a role at Simulon, please reach out to us by email.
The Inspiration For Simulon
The inspiration for Simulon came from my lifelong fascination with visual effects, sparked by iconic films like the first Jurassic Park. The project is a huge undertaking, and has been in development for four years. We aim to enable indie creators to produce immersive, photorealistic VFX content of their own without needing huge budgets or complex setups. Our target audience includes independent 3D artists, content creators, and small studios who aspire to bring their creative visions to life with high-quality visual effects. In 2020, I shared the first video made with an early prototype of Simulon, and the influence of Jurassic Park is quite evident in that initial showcase:
Development & Features
Simulon is an intuitive, end-to-end solution for crafting high-quality VFX shots. It bridges the gap between real-time AR and high-fidelity offline rendering and compositing. Our approach is to consolidate what used to be a number of fragmented processes into a single, integrated experience. This not only makes the process more enjoyable but also allows creators to focus on the creative aspects of visual effects.
With our desktop application, creators can upload 3D assets in various formats like glTF, USDZ, or even full Blender files, complete with intricate systems and shaders. Our uploader intelligently creates linked variants of these assets, optimized for quick downloading and performance on mobile devices while achieving uncompromised quality in the cloud and on the desktop. It maintains the integrity of the assets by syncing transformations and animation states with the high-quality versions used in the final rendering.
On the mobile side, creators can access their own assets along with a shared library, enabling them to place and compose these assets in a real-time AR session. Before composing their scene, creators capture their environment in 360 degrees using our intuitive capture workflow. Our machine learning algorithms then convert this capture into a 360 HDRI for accurate image-based lighting, providing a real-time preview on the device to visualize and adjust the lighting if necessary.
Our system also dynamically emulates camera response functions, allowing the virtual camera to adjust exposure in tandem with the real camera sensor. Creators can record multiple takes within their composed scenes, organized through our project management system. Each scene retains data about the physical location, so creators can return to the exact spot and find their assets positioned as they left them.
The real magic happens when the captured takes are rendered through our distributed cloud rendering service, resulting in photorealistic, fully composited VFX videos. For those familiar with tools like Blender, our desktop app allows for local rendering and further refinement of scenes with an automated scene graph, compositing nodes, and synced camera and asset animations. Changes made in Blender are updated globally, enabling a unique remote authoring workflow for live-action locations. Editing scenes becomes intuitive as creators have access to a reconstructed mesh of the real-world scene for reference. This unique workflow allows for context-sensitive editing and enhances the creative process, making Simulon a fundamentally different tool in the realm of VFX and AR.
Setting Up the Mosquito in Amber Demo
The Mosquito in Amber demo was a showcase of Simulon's non-destructive integration with traditional 3D pipelines. I started by using Simulon to capture the environmental lighting and position the amber model in the scene for a live video shoot.
This initial footage and the 3D scene were then imported into Blender using our desktop app. Our app's automatic camera tracking and field of view setup made it straightforward to use KeenTools GeoTracker to correctly track a real-world moving object (a second iPhone) that also respected the camera motion. By attaching a 3D light source to this second iPhone in Blender, I was able to dynamically illuminate the amber, creating a believable play on light.
Utilizing GeoTracker
KeenTools' GeoTracker plug-in is a powerful tool that allows for precise object tracking in Blender. You align the 3D object from the camera's perspective to a matching object in the real world and then let the plug-in track it through the subsequent frames. Some manual adjustments are needed if the track gets lost, but overall, the process is efficient. For the Mosquito in Amber demo, GeoTracker was used to track the moving iPhone so that it could be used as a dynamic light source.
Promoting Simulon
The approach to promoting Simulon has been quite organic. We've been sharing glimpses of our journey through development videos, giving a sneak peek into what Simulon is capable of. But what we're really excited about is the story that's yet to unfold.
Our focus is on fostering a space where creativity flourishes in ways that were previously unimagined. A place where the boundaries between creating and sharing become seamless, where every artist, big or small, has the tools and support to make their vision come alive. We envision a platform where 3D Artists can distribute digital assets that spawn new UGC universes with stories and content that is authored by the community.
Currently, we're working closely with a select group of artists and studios as part of our preparations for next year's launch. This collaboration is a step towards expanding our community outreach, which we'll be announcing more about on social media soon. For 3D artists and studios interested in joining this creative venture, whether it's to feature or sell their assets on Simulon or to talk to us about other forms of collaboration, we encourage you to reach out to us.
Future Plans
As we look to the future, our vision for Simulon is not only to continue to advance technologically but also to foster a thriving community of independent creators. For example, a key initiative in this direction is the establishment of a Creator Fund. This fund will be designed to provide vital support to indie projects, as we are deeply committed to seeing independent creators flourish beyond the traditional big studio environment.
In terms of technological innovation, there is not much that I am able to share at this point beyond the fact that we have a number of incredibly exciting developments in progress. Our promotion strategy focuses on sharing real-world examples and testimonials from creators using our tools. We believe this authentic approach will resonate well with our audience. Simulon is set to be publicly available early next year, and those interested in our closed beta can apply here. We'll be reaching out to applicants soon as part of our phased beta release.