NVIDIA has its fair share of closed-sourced trade secrets that power their AI business empire, but on the other hand, they are also helping the industry thrive with many projects that are open-sourced. Today, they have moved something from close to open, as they chose the Audio2Face technology so that everyone can start using it to build life-like characters and AI avatars.
For recap, Audio2Face analyzes the audio’s phonemes and intonation, turns that into animation data, and then maps it to a character’s face. Developers can use it either offline for pre-scripted scenes or stream it live for dynamic, AI-driven interactions. By going open source, NVIDIA is giving developers, students, and researchers the ability to dig into the models, SDK, and even the training framework – meaning anyone can fine-tune it for their own characters and use cases.
Tons of studios are already using it, from Convai and NetEase to Codemasters, GSC Games World, Inworld AI, Perfect World Games, Streamlabs, and UneeQ Digital Humans. Reallusion, which makes tools for building 3D characters, has integrated Audio2Face into its platform, while game developers like Survios and The Farm 51 have been using it to speed up production and push realism in their titles.
Team Green is hoping this open source release sparks a bigger community around AI-driven animation, as they’ve even set up a Discord for developers (Click Here to Join) to share their work. With this move, it looks like lifelike, AI-powered avatars will be everywhere and anywhere when there’s a need for NPCs to talk in games, media, or beyond.
