Neural Networks and AIs are a New Tool for Virtual Platform Developers

Neural networks have become a new technological trend, which is already being used as a tool by some specialists. It will be possible to notice how neural networks have influenced the development of games, apps and virtual platforms in the near future.

We talked with a game designer named Max about the prospects for using neural networks in game development. He offered us various options for using neural networks, which we will discuss in the article.

Prospects for the Use of Neural Networks

More than a thousand neural networks and hundreds of their applications appeared in 2023. The network community even revealed the opinion that they will soon replace human specialists. But the neural network itself cannot create a product without a person. It needs a command and an estimation of its results. It’s not enough to be able to manage a neural app, you need to be an expert yourself for the professional use of a neural network.

That’s why neural networks will not replace developers, but will become a tool in their hands. For example, they will simplify, automate and speed up certain workflows. In the same way, designers prefer to use Adobe programs with an increased and expanded set of tools now.

If you learn how to use a neural network, you can get results that you can already take into work with minimal editing or even without it. That’s why another skill can be added to the developer’s portfolio — the ability to work with a neural network.

Ways to Use Neural Networks

Neural networks are popular for creating text and images now. But these are not the only possible ways to use them. The game designer identified 7 options for using neural networks that can be used in project development now or in the near future.

1) Neural networks for coding. They already know how to analyze the program code and offer their own options for changing it based on a request from the developer. They can also add lines of code based on the available data.

The results of the neural network’s work with the code cause a lot of controversy among users so far. Neural networks haven’t yet learned how to write optimal code options and often offer outdated options instead of optimal ones.

But there are neural networks that can be integrated into game engines. This is more convenient because it’s easier for the program to parse the code. In addition, the developer has the opportunity to immediately estimate the code’s architecture, without the need for manual transfer from third-party websites and further uploading into the engine. For example, OpenAI GPT-3 works with the Unity engine.

2) Neural networks for design development. Regular users are already using neural networks to transport their favorite characters to different worlds using Midjourney. They change the setting, genres, eras and styles using prompts. 

And this tool will be able to create well-defined images for games and apps in the hands of a designer. For example, a neural network can easily generate character design, environmental design, and user interfaces.

3) Neural networks for creating 3D models. 3D modelers can also simplify their work with the help of neural networks. For example, they can create 3D models from 2D images or text prompts. Assets Scout works with images, Luma AI Imagine 3D works with text.

Only a small number of neural networks can generate pro level models so far. And it takes a lot of time — from half an hour to several hours to create a single 3D model. But it’s still faster than working from scratch.

Some neural networks can still generate models relatively quickly. But at the same time, their quality is still too low to be used in professional development.

4) Character animation. You no longer need to manually set up all the motions or spend money with motion capture. There are several neural networks that capture human movements from a video or in real time and transfer the animation from the video to your 3D model.

The animations made by such neural networks are highly dependent on the quality of the video. And the resulting animation will still need to be finalized manually. But even in this form, neural networks are already saving developers both time and money.

5) Neural networks for audio synthesis. Studios hire a composer, a sound engineer, or a sound designer to create audio for games and metaverses. And neural networks cannot fully replace their work yet.

But some audio, like sounds of nature, the rustling of clothes, the steps of characters or simple music can be created using a neural network. Programs can even use voices with given characteristics. Although they do it so far much worse than professional actors, but this option for creating audio accompaniment is suitable for small development teams, startups and indie developers.

6) Neural networks for creating text. Developers work with texts for the interface, descriptions of objects and locations. Particular attention is paid to the scripts and character dialogues. And all this can already be generated by neural networks.

Programs can analyze the context, adopt the style of the author, and write their own scripts and books. For example, several short films and YouTube videos based on the ChatGPT’s scripts have already been shot. But the neural network can also be used for more complex projects. 

Examples of Using Neural Networks in Gamedev

You can already find examples of the use of neural networks in game development. Their creators talk about the stages of their work and impressions from the use of neural networks.

For example, the Anticthon game developer added several neural networks to the code at once. The user can communicate with NPCs without using voice chat only.

The developer integrated the Open AI GPT-3 neural network into the game’s NPCs. They can perceive the player’s speech and analyze it in order to conduct dialogues right during the gameplay. And so that the character can respond, voice synthesis is configured using Google TTS. And the sound of voice is also hand-picked by the developer for each character specifically.

Another developer used Midjourney when working on the character and location design. He showed the stages of working with the algorithm along with the selection of images and transferring a 2D image to a 3D model. According to him, he was able to save time when working on images: 2 days of work on the character and 3 days on the location with the neural network.

And Microsoft have tested the OpenAI Codex neural network based on the GPT-3 algorithm in the Minecraft metaverse. The developers tested the possibility of building objects in the virtual world based on text requests. The neural network was able to generate models of the house, animals and characters right in the game in real time.

TL;DR

Neural networks give developers new opportunities to optimize their processes. And there will be even more options for their use with the development of the technology.

Now there are several ways to use neural networks in development:

• coding;
• designing characters and locations;
• creating 3D objects based on text prompts and images;
• character animation based on video;
• synthesis of audio effects to fill the world and voice characters.
• creating text for the script and NPC’s dialogues.

---