#MELODY ASSISTANT ADD LYRICS TO VOIC HOW TO#
#MELODY ASSISTANT ADD LYRICS TO VOIC GENERATOR#
#MELODY ASSISTANT ADD LYRICS TO VOIC SOFTWARE#

Now that I have a demo version, I can work on making improvements to it. I knew ahead of time it was not going to sound very good, but this is just a start. Not just for music like everyone else is focusing on, but with lyrics and vocals to make it into a real song. In the end, I accomplished my goal, which was to set up an automated framework for creating an AI song with no human intervention needed. I did this by converting the MIDI file to WAV, and then using ffmpeg’s amerge command like this: ffmpeg -i imabandoned.wav -i imabandoned_singer.wav -filter_complex amerge=inputs=2 -ac 2 imabandoned_final.mp3 The final step was to combine that voice file with the original MIDI music file to produce the song.

Next, I used an open-source virtual singer program named midi2voice to create the vocals (a WAV file), using the lyrics and music as input. I used an open-source program called banana-split to get this done. First, I had to convert the original MIDI file into a new MIDI with only one channel containing the piano part, because that is usually the melody the singer needs to follow. There was also an issue that I couldn’t just simply feed the MIDI music file to the vocal synthesis program directly. Then on top of that, it has to try to figure out exactly where the virtual singer should say each word, to stay on beat. It has to detect the pitch of the melody in the music, convert text to voice, and adjust the voice to match the notes of the melody. The vocals were one of the hardest parts. And the song title was chosen based on various non-AI rules, mostly relating to what line is used the most in the lyrics.

#MELODY ASSISTANT ADD LYRICS TO VOIC GENERATOR#

The lyrics were written using the open-source GPT-2 Simple natural language program, which I trained on poems and song lyrics as described in my article how I created a lyrics generator about this lyrics site. Specifically, I used the pre-trained “trio” model, as described in their MusicVAE blog posting. The music was automatically generated with Google’s open-source Magenta. You can listen to 10 of these songs, and also view the lyrics, on my AI Music Site. What matters is that it is one of the few songs I know of that was 100% made using AI from start to finish.

And what I have talked about so far does not even get into all the issues involved with having the AI write lyrics and sing the song.įor comparison purposes, here’s my AI masterpiece, where the computer created everything (song title, lyrics, melody, music, and vocals): You would never confuse it with a real song you might hear on the radio.īecause of all of this, AI music companies and AI bands generally use a mixture of human input and AI input (see some good examples in this Google blog posting). And, making things worse, MIDI files most of the time have a very computerish/fake sound, like old video game music.

#MELODY ASSISTANT ADD LYRICS TO VOIC HOW TO#

It does not have a good sense of the beat and how to mix various instruments together to follow the piano melody. The problem comes when you start trying to have the computer automatically add other instruments to it. The main output of the AI is a melody, usually in the form of a MIDI piano track.

#MELODY ASSISTANT ADD LYRICS TO VOIC SOFTWARE#

They usually use music editing software to vastly improve on what the AI comes up with. But, what you may not realize, is that the original version of the song does not sound anything like what you hear. And there are also AI bands releasing great music online using similar machine learning programs. This amazing piece of AI music was created by a company named Aiva, which uses cutting-edge machine learning technology to create tunes like that.

Melody assistant add lyrics to voic