Singing Sophia: an AI Voice for a Humanoid Robot
Built the neural singing voice for Sophia the Robot, enabling her to sing from sheet music. The voice debuted in a duet with Jimmy Fallon on the Tonight Show.
Debuted in a duet with Jimmy Fallon on the Tonight Show. Enabled Sophia to sing a rendition of nearly any song from notes and phonemes.
Sophia's singing voice was built in a series of steps. A vocalist recorded a set of passages in the studio, and those recordings were fed into a neural network that learned the content and contours of the performance.
With the trained network, a sequence of musical notes and vocal phonemes could be turned into a signal the listener perceives as human singing. In effect, Sophia could read sheet music and sing almost any song given the right words and frequency values. Frequency and amplitude modulation added a natural vibrato that brought the performance to life.
