Anymore
Anymore - Social Content Platform
NVIDIA Audio2Face technology uses artificial intelligence to automatically match sound with moving faces.
NVIDIA Audio 2 Face is an impressive tool that runs on Omniverse, the company’s open real-time simulation platform. The technology is capable of applying animations that match surprisingly well to the Digital Mark 3D character model, which is included with an audio file. Whether you’re animating characters in games, movies, real-time digital assistants, or just for fun, technology can simplify the animation of a 3D character to fit any voiceover track. The technology does this automatically, works well in most languages, and lets you apply more details.
To do this, NVIDIA has implemented a deep neural network that maps facial animations to audio in real time. When you first start the program, it may take some time to build the Tensor RT Engine, which optimizes the neural network for your hardware. From there, you can see and act on changes in real time as long as your hardware can handle it.
As an added bonus, the company’s educational website features loads of videos for NVIDIA on Demand, Omniverse, and Audio2Face. The first video on Audio2Face, even detailed information on how to export the Process to other tools like Unreal Engine 4 is available here.
The success of the process also seems to depend on the quality of the sound used. Charlie Chaplin’s unusually famous speech in the movie The Great Dictator is featured in a video, and the movements don’t look as clear as they do on more lively, modern recordings. However, unlike the sound quality, we can easily say that this vehicle will get better over time.
NVIDIA Audio 2 Face Converts Voices to Facial Mimics
Williams Johnson