Meta this week previewed a voice-based generative AI model that could one day swap your virtual assistant’s voice for the voice of someone you know.
“Voicebox can produce high-quality audio clips and edit pre-recorded audio — like removing car horns or a dog barking — all while preserving the content and style of the audio,” Meta says. “The model is also multilingual and can produce speech in six languages.”
Meta teased Voicebox as a way to make virtual assistants sound less robotic or power non-playable characters in the metaverse. But for now, we’re just getting a sneak peek.
“Because of the potential risks of misuse, we are not making the Voicebox model or code publicly available at this time,” the company says. “While we believe it is important to be open with the AI community and to share our research to advance the state of the art in AI, it’s also necessary to strike the right balance between openness with responsibility.”
To that end, Meta’s AI team shared audio samples and a research paper that details the results they’ve achieved thus far.
In a video demonstrating text-to-speech capabilities, we see an audio clip run through Voicebox produce the same phrase in six different voice styles. It also takes a clip of someone talking and has their voice read a different phrase they never uttered in real life.
It also stripped out audio of a dog barking in the background of meeting, and changed a word (“guys” to “everyone”) in the final, recorded version. And, it took audio of someone saying something in a different language and had that voice say it in English, using their voice style.
The news comes several months after CEO Mark Zuckerbeg said the company was "creating a new top-level product group at Meta focused on generative AI to turbocharge our work in this area." Meta already had several teams devoted to generative AI, but it merged those teams into one group focused on bringing “delightful experiences” to Meta’s various apps and services.
One experience that may or may not be delightful: AI-generated ads. Meta is also testing the use of AI algorithms to help advertisers create text and images for online ads.
Whatever the company comes up with, Meta is reportedly playing catch-up on AI. As The Wall Street Journal reports, Meta is “scrambling” to come up with AI products that can generate revenue “after spending years prioritizing academic discoveries and sharing them freely.” One roadblock: many of those working on AI have left the company in the last year, the Journal says.
In the interim, Microsoft has dumped billions into ChatGPT creator OpenAI and overhauled its Bing search engine thanks to an AI chatbot integration. Google released Bard and is experimenting with a Search Generative Experience that threatens to upend search as we know it.
