Teaching a Robot to Play Jazz

For a long time I carried a very specific dream: to have a robot trained well enough to play jazz, and not just play it, but improvise, live, alongside a human band. Jazz is the hard case because it is not about executing a score. It is about listening and responding in the moment. If a robot can do that, it is doing something that looks a lot like creativity.

I got my chance at a global conference in Chicago. I hired a local jazz trio for the gig, and during rehearsal I recorded everything the pianist improvised. That recording became the training data. We taught Baxter, our collaborative robot, from the pianist's own playing, and then we put it on stage to perform live in front of more than five hundred people.

What moved me was not that the robot hit the right notes. It was the moment of exchange, the sense of a machine taking a turn in a conversation that humans have been having for a century. I have spent my career at the seam between the physical and the computational, and this was one of the purest examples of it: acrylic, motors and code, swinging.