You can better control diffusion models without leaving Blender.
Creator coolzilj has released a handy script that allows you to use the neural network ControlNet inside Blender.
ControlNet lets you control text-to-image diffusion models by adding extra conditions, which makes generated pictures more accurate, like these realistic renders of GTA characters.
To use the script, you will need the AUTOMATIC1111 Stable Diffusion web UI and Mikubill's UI for ControlNet.