3D Asset Generation: AI for Game Development #3

Welcome to AI for Game Development! In this series, we'll be using AI tools to create a fully functional farming game in just 5 days. By the end of this series, you will have learned how you can incorporate a variety of AI tools into your game development workflow. I will show you how you can use AI tools for:

1. Art Style
2. Game Design
3. 3D Assets
4. 2D Assets
5. Story

Want the quick video version? You can watch it here. Otherwise, if you want the technical details, keep reading!

Note: This tutorial is intended for readers who are familiar with Unity development and C#. If you're new to these technologies, check out the Unity for Beginners series before continuing.

Day 3: 3D Assets

In Part 2 of this tutorial series, we used AI for Game Design. More specifically, we used ChatGPT to brainstorm the design for our game.

In this part, we'll talk about how you can use AI to generate 3D Assets. The short answer is: you can't. That's because text-to-3D isn't at the point it can be practically applied to game development, yet. However, that's changing very quickly. Keep reading to learn about The Current State of Text-to-3D, Why It Isn't Useful (yet), and The Future of Text-to-3D.

The Current State of Text-to-3D

As discussed in Part 1, text-to-image tools such as Stable Diffusion are incredibly useful in the game development workflow. However, what about text-to-3D, or generating 3D models from text descriptions? There have been many very recent developments in this area:

• DreamFusion uses 2D diffusion to generate 3D assets.
• CLIPMatrix and CLIP-Mesh-SMPLX generate textured meshes directly.
• CLIP-Forge uses language to generate voxel-based models.
• CLIP-NeRF drives NeRFs with text and images.
• Point-E and Pulsar+CLIP use language to generate 3D point clouds.
• Dream Textures uses text-to-image to texture scenes in Blender automatically.

Why It Isn't Useful (yet)

The Future of Text-to-3D

Attribution

Thanks to Poli @multimodalart for providing info on the latest open source text-to-3D.