Developing a game making use of ai 3d model generation
The idea:
To use AI Generative Models to supplement a lack of 3D modelling ability to allow a solo programmer or developer to create a functional prototype of a game more effectively.
Abstract
The focus of this project is to create a prototype of a game, using AI 3D model generation to supplement a lack of modelling skill and allow a solo developer or small team of programmers to create a functional prototype that conveys the idea and aspect of the planned game. This project was a success Text-to-3D and Image-to-3D models can be used effectively to supplement missing skills in a development team, however the models generated are not always usable, around 1/4 are good and usable. On top of this if a developer possesses basic modelling ability, they would be able to greatly improve this ratio by tidying up and fixing the broken models in software such as Blender or Maya. The prototype developed within this project is very much playable and looks significantly better than simple Unity geometry and programmer art.
This project was created using a combination of Python and Unity with C# scripting
Background:
Text-To-3D and Image-To-3D Model generation is the back bone to creating this project. Both of these methods work using various combinations of diffusion models, nerf models and some other more niche methods. Point-E Was used for this project which makes use of a glide model and two diffusion models
Design:
The idea was to create a wave based tower defense game using as many or if possible solely ai generated 3D models. The reason for this choice is that it is relatively simple but allows for many different kinds of models to be used allowing for an effective test of Point-E across multiple uses.
Implementation:
There were a few steps to implementing the design, the first being to learn how to effectively use Point-E, this mostly consisted of changing variables and settings and generating new models over and over. Then it was time to build up the actual game to which there was a few key scripts that needed to be made.
Turret script
Enemy script
Player stats
Building menu
UI
Conclusion:
The experiment was a success, Point-E or other AI 3D model generation can be used to prototype a game effectively as a solo developer who lacks the skills to create them manually, however a lot of models went to waste and with them a lot of time spent generating them. So, while it worked 3D generative AI still has some way to come, that being said if the developer has some level of modelling skill even if it is basic the models generated could be cleaned up to look much nicer with very little effort.
If you would like to read my full dissertation paper that goes into more detail click here.