Create a 3D World with AI-Depthmaps

AI-generated depthmaps are becoming increasingly popular in creative workflows. Depthmaps are grayscale images that represent the depth of objects in a scene. Using machine learning algorithms that analyze an image, state-of-the-art AI-tools are able to quickly and simply generate a depthmap that can be used in a variety of ways. In this blogpost, I will show you a simple but impactful use-case for using depthmaps to create 3D perspectives.

Using AI-Depthmaps in Your Work

In digital art, depthmaps enable artists and designers to produce stunning 3D effects that add depth and realism to their work. With depthmaps, you can finely tune the depth of each element in the scene, a feature that would be almost impossible to achieve manually.

Meanwhile, in video production, depthmaps are often used to create depth-of-field effects in post-production. Filmmakers can blur certain areas of the image to draw attention to a specific subject or create a more cinematic look. Additionally, depthmaps can help separate foreground and background elements, making it easier to apply visual effects or other adjustments to the scene. Whether you're a digital artist, a designer or a filmmaker, AI-generated depthmaps offer limitless creative possibilities.

AI-generated depthmaps can also be used in the field of virtual and augmented reality. By using depthmaps to create a 3D model of a scene, developers can create more realistic and immersive virtual and augmented reality experiences. This can be particularly useful in applications like gaming or training simulations, where users need to interact with a virtual environment in a realistic way.

How To Create The Illusion of Depth & Perspective in a 3D Scene

There are multiple ways to achieve a depthmap along with your AI-generated output in AUTOMATIC1111’s WebUI. You can use the depthmap-script (available in the official library of the installable extensions) to transform your prompt or your image into a dephtmap. This can also be easily done using the ControlNET model for depthmaps! Explore the possibilities in A1111’s WebUI, there is also an extension called ‘Corridor Crawler Outpainting’ that lets you generate hallways with the depth-to-image model. The ‘depthmap2mask’ extension creates masks for img2img based on a depth estimation made by MiDaS.

AI-generated depthmap of a scene

Examples of output generated with corresponding depthmap

When you're happy with your scene, you can transform it into a 3D world with the help of free to use and open-source software Blender. You need to install and use the plugin 'fSpy' to create the illusion of perspective with the help of your depthmap. Upload the image and manually adjust the vanishing point axes to take control over the right perspective. Then import the file into blender to get a 3D environment where your image can be projected back onto the scene via UV-projection! Here is an example of how the depthmap made a walking animation possible and believable:

2D to 3D with AI-Depthmaps

Whether you're a digital artist, a designer or a filmmaker, AI-generated depthmaps offer limitless creative possibilities. They are a versatile tool that can be used in a variety of creative workflows. It's worth exploring the capabilities of AI-generated depthmaps and thinking about how they can be used to enhance your creative work!