LogInKristof Beets from Imagination Technologies discussed the mobile developments, talked about its differences from creating a game for PC/consoles, and shared how mobile developers solve various production problems in terms of CG-art and optimization.
Introduction
I’m Kristof Beets, Senior Director of Technical Product Management here at Imagination technologies. I have worked for over 20 years in the IP development space working on GPU technology across the full spectrum from general-purpose to high-end gaming. In this time, I have worked in developer support, demo development, and also business development.
I have been in the mobile GPU space since its inception and have also been with Imagination since the beginning of their ray tracing development dating back to 2014 with our Plato Boards. I am now in charge of heading up power-efficient IP to deliver ray-tracing to a wide variety of markets and platforms.
Imagination Technologies have been an IP creating company for over 35 years, in that time, we have been at the forefront of graphical technology. We specialize in the mobile, embedded, and cloud computing space currently, working out how to bring the best gaming experience to new platforms through intelligent hardware design.
What are the Differences Between Mobile Development and Creating Games and Other 3D Projects for PC/Consoles?
Power Budget!
We do not have 200+ Watt of power/thermal budget to play with, in mobile, the GPU power budget is generally around 1W or at most 2W (this is less than 1% of a desktop GPUs power budget) hence our focus is how to deliver more for less. Consoles and PCs not that long ago would deliver 1080p games and today phones almost always have a 1080p screen or even higher, and we also see higher than 60Hz refresh rates, so the spaces are coming closer together.
Power/Thermals also limits bandwidth, in mobile, we simply cannot use LPDDR5/6 memory at ever wider or higher speeds as the power goes up with every pin, HBM again is also much higher in power cost so would simply not fit in the thermal budget of any phone.
When creating content, this mean developers should also be aware of this reality, e.g., if we make lots of effort to make the HW and Driver software ultra-efficient we cannot have a developer submitting the whole world geometry and expecting the GPU to just deal with it, the game engine has to embrace the same efficiency concepts for best results. Basically, the PC world can be very forgiving as it's all very brute force with lots of processing and bandwidth, extra polygons to render, or a higher resolution texture to load won’t always heavily impact performance. Small mistakes in mobile, however, can result in catastrophic battery life or performance experiences so optimization is key.
Main Limitations and Challenges of Mobile Development
In reality, the budgets for GPUs have been going down since everything we add has a cost, and also form factors and resolutions/refresh rates are ever higher, basically, that faster 5G modem also uses more power than AI Core, they added costs more power/bandwidth.
This puts us as a GPU IP developer in a bind, more and more is required of a GPU to power 4k HDR high refresh phone displays, but the budget for the GPU has gotten smaller. Process technology has helped a lot but creativity on our IP design is just as critical to enabling ever more for less. By squeezing out more power from the limited GPU budget, we build the groundwork to allow developers and artists themselves to not have to cut even more corners than are required to get great games running on the go.
Tools Used for Mobile Development
Indeed, internally we use all common tools to develop demonstrations and test and improve our hardware and drivers. The game engines have many profiles and modes which we help to optimize for mobile and, in many cases, they are based on techniques first developed on PC or console. UE4, for example, allows you to set up your project to be optimized to run on mobile at the outset.
We also offer many free tools that help developers do bottleneck analysis and this includes full hardware profiling tools as well as step-by-step debuggers supported by the hardware. This means that developers can easily and rapidly test their games and artwork on a variety of simulated platforms. This means that they can find out if an effect, a particle system or even something as simple as a high-resolution texture is causing their games to slow down.
We also back this with SDKs and example code as well as support forums and portals for mobile developers. There are also direct contacts and training courses available to help you get the most out of your game or application on mobile platforms.
Art Challenges
Both Unity and Unreal include full workflows to handle this, they can tune the artwork complexity down as well as modify the rendering quality and effects specifically for mobile.
If a developer is using a custom in-house engine developed for say DirectX, then the porting effort would be much more extensive. Some of the key areas that will really slow down something on a mobile device is texture quality and shader complexity. Console and desktop quality hero models can have multiple 4k texture sheets with transparency and translucency. For mobile, these need to be reduced for the sake of performance, but also because on a 4-6” display, a lot of that super fine detail will be imperceptible to the player anyway.
Something else to consider is the normal maps used for creating surface detail. On a smaller screen, the macro detail needs to be emphasized over the micro-detail. A smaller screen space allows for the detail to be more readable and makes material types more identifiable.
Whilst mobile GPUs have gotten better at rendering more triangles on the screen, the model may still need to be further optimized and detail that would traditionally be geometry would need to be captured in normal maps (which will pop more, thanks to our mobile ray tracing algorithms coming soon.)
Creating Detailed Landscapes for Mobile
Many of these elements use LOD techniques, vegetation is often to some degree procedural hence the developer can enable more/fewer instances of such vegetations or different Levels of Detail (LOD) of the objects themselves. Even on PC/Desktop and Console such techniques are needed, for example, PS5 versus PS4/3/2 often games continue to support multiple generations, or even in the case of XBOX, there are 2 performance points with the X and S Series versions, so mobile is just the same, it's yet another version/performance point to handle.
VFX Production
This one again is fully procedural and done in code, so tuning the particle counts to a device is in a way easier to reduce the load but the setting may need some experimentation work to obtain similar effects. For example, rather than each particle being its own rendered point in the VFX sim, using larger more geometrically simple cards could be one way to save.
This is where the technical artists really come into their element. They can spend the time bridging the gap between the hardware limitations and the project's visual aspirations to tune the particle effects controls to obtain the right balance in complexity and visual quality.
Mobile Games Optimization
We see a wide range of controls, a game such as PUBG today on mobile alone supports 5 quality levels. This will change the resolution (e.g. render at a low resolution than the display and upscale it, maybe in the future this will include effects/processing such as Super Resolution). They will also target different FPS to match the refresh rate of the phone's display. There are also different quality options to cover things like HDR, changes in texture filtering from bilinear to trilinear to anisotropic and MSAA from 0x to 2x to 4x. In many ways, the mobile market is becoming similar to that of the PC market.
In the PC market, we see tons of options because everyone’s hardware is different, some might be slower or older, whilst others will be the latest and greatest, but everyone wants to play. With the wide range of smartphones and tablets now available, the approach has become the same in allowing users to control the quality of their titles to get the best experience.
Some developers will run mini-benchmarks to judge the capabilities of the platform, some devices have extensions that provide feedback on the thermal state so the game engine can adapt in real-time, some developers have lists of devices that provide for certain quality levels (certification) so we see lots of different approaches and mechanisms and all of the strengths as well as weaknesses and our DevTech team is always available to help.
Future of Mobile Games
Flexibility will be key as the mobile market goes from 1 pixel/clock low-end devices to TerraFlops devices which will beat low-end PC cards. This means IP vendors like us need to be designing for flexibility and scalability going forward – we should also consider mixed-mode rendering or the cross-over from mobile to cloud e.g. games may at some point render fully or partially in big servers in the cloud.