In January, Google made some waves — sound waves, that is — when it quietly released some research into its AI-powered music-making software that created tunes based on word prompts. Today, its sister company Google DeepMind went several steps further — it has was announced a new generation model of music called Lyria this will work in conjunction with YouTube, and two new toolsets he describes as “experiments” based on Lyria: Dream Track lets you create music for YouTube Shorts, and Music AI is a set of tools he says aims to aid in the creative process (for example, creating a melody from a track that a creator might hum). Along with these, DeepMind said it is adapting SynthID – used to tag AI images – to also watermark music with AI.
The new tools come at a time when artificial intelligence continues to stir controversy in the creative arts world. It was a key issue at the heart of the Screen Actors Guild strike (which finally ended this month). And in music, while everyone knew Ghostwriter was using AI to emulate Drake and The Weeknd, the question you have to ask is whether AI creation will become more of the norm in the future.
With the new tools being announced today, the first priority for DeepMind and YouTube seems to be creating technology that helps AI music remain credible, both as a complement to creators today, but also just in the more aesthetic sense of sounding like music.
As Google’s previous efforts have shown, one detail that often comes up is that the more one listens to AI-generated music, the more distorted and surreal it starts to sound, moving further away from the intended effect. As DeepMind explained today, this is partly due to the complexity of the information contained in musical models, covering beats, notes, harmonies and more.
“When generating long audio sequences, it is difficult for AI models to maintain musical continuity in phrases, verses or extended passages,” DeepMind noted today. “Since music often involves many voices and instruments at once, it is much more difficult to create than speech.”
It is notable, then, that some of the earliest applications of the model appear in shorter pieces.
Dream Track is initially released to a limited set of creators to create 30-second AI-generated soundtracks in the “vocal and musical style of artists such as Alec Benjamin, Charlie Puth, Charli XCX, Demi Lovato, John Legend, Sia, T-Pain, Troye Sivan and Papoose.”
The creator enters a theme by choosing an artist. A track with lyrics, backing tracks and the featured musician’s voice are used to create the 30 second track, which is intended to be used with Shorts. Here’s an example of a Charlie Puth track:
YouTube and DeepMind clearly point out that these artists are involved in the project, helping to test the models and providing other input.
Lyor Cohen and Toni Reed, respectively head of music for YouTube and vice president of emerging experiences and community projects, note that all of the Music AI tools being released come from the company’s Music AI Incubator, a group of artists, songwriters and producers working in tests and give feedback on projects.
“It was clear early on that this initial group of participants was intensely curious about AI tools that could push the boundaries of what they thought was possible,” they note. “They also looked for tools that could enhance their creative process.”
While Dream Track is in limited release today, the Music AI tools aren’t due until later this year, they said. DeepMind teased three areas it will cover: creating music on a specific instrument, or creating an entire set of instruments, based on humming a tune. using chords you build on a simple MIDI keyboard to create an entire choir or other ensemble. and creating backing and instrumental tracks for a vocal line you may already have. (Or, actually, a combination that uses all three, just starting with a simple hum.)
In music, Google and Ghostwriter are of course not alone. Among other tools being released, Meta created an open-source AI music maker in June. Stability AI released one in September. and startups like Riffusion are also raising money for their efforts in the genre. The music industry is also trying to prepare.
