It has been more than half a year since Google Inc. first revealed a piano-playing artificial intelligence that can take a few notes and turn them into a song. Now the search giant has decided to share the experience with the world with its AI Duet web app.

Yotam Mann is the musician and coder behind the new AI, which he created with help from Google’s Magenta and Creative Labs teams. In a video, Mann explained the process behind the AI and how it manages to play along with new music on the fly.

“Making music using code isn’t a new thing at all, but machine learning gives us a different way to go about it,” said Mann. “If I was trying to make AI Duet with more traditional programming, I’d have to write out lots of rules.”

Mann explained that using older methods of AI programming, he would have had to tell the computer exactly how to respond to an impossible number of inputs. “I’d basically be creating this map to tell the computer how to make these decision, but there are just too many note and timing combinations to map it all by hand,” he said.

Rather than programming AI Duet with specific rules for how to respond to notes, Mann trained the AI using neural networks. This technique attempts to work roughly like the human brain, with artificial neurons working together to form links and reinforce existing knowledge.

Google fed its AI a massive number of note combinations, and from that data, the AI learned how to form connections between notes that are often played together or in sequence, their timing and so on. With AI Duet, when you play a few notes, the AI compares that data to the links created in its neural nets and then decides what to play from a few different possibilities. Mann said that the AI will try to make sense of any notes you give it, even if you just mash the keyboard, but it will also try to follow along with rhythm and melodies.

Mann built AI Duet using Magenta, an open-source deep learning library that the Google Brain team built with TensorFlow. The code for AI Duet is also open source, so if you want to build your own piano-playing robot, you can get started now.

My results with Google’s AI Duet have not been particularly catchy so far, but then again, maybe the AI is only as good as its partner. Either way, it still won’t play “Chopsticks” with us.

Here’s the video:

Image courtesy of Google