Pictionary is a pretty simple game for humans, who are challenged to draw a picture based on the cue card they receive. Now, thanks to Microsoft Corp., the game is no longer that difficult for computers either.

Microsoft said today it has developed a new artificial intelligence-based bot that can draw images based on text descriptions. The “drawing bot” can generate images using caption-like text descriptions as the only input.

With this method, Microsoft said, it has been able to achieve a threefold improvement in image quality compared with older text-to-image techniques. In addition, Microsoft’s drawing bot can even imagine extra details not included in the original text description, thanks to its “artificial imagination.”

“If you go to Bing and you search for a bird, you get a bird picture,” Xiaodong He, research manager at Microsoft’s Deep Learning Technology Center, wrote in a blog post. “But here, the pictures are created by the computer, pixel by pixel, from scratch. These birds may not exist in the real world — they are just an aspect of our computer’s imagination of birds.”

Microsoft’s drawing bot uses a technology called the Generative Adversarial Network, which relies on a pair of machine learning models. The first of these can draw images based on text descriptions, while the second – called the discriminator – is used to determine the authenticity of the images the bot creates. “Working together, the discriminator pushes the generator toward perfection,” Microsoft explained in a more detailed research paper.

After being fed with enormous data sets for training purposes, the drawing bot began to come up with its own ideas about what’s right and wrong, Microsoft said. For example, the bot understands that birds often perch in trees. Because of that, whenever it was asked to draw a bird from the text description, the bot would use its “imagination” to depict that bird gripping a tree branch even though there was no mention of this.

Microsoft also had some fun feeding the bot with text descriptions that are counter to its training. In one test for example, they asked it to draw “a red double-decker bus floating on a lake.”

“It generated a blurry, drippy image that resembles both a boat with two decks and a double-decker bus on a lake surrounded by mountains,” Microsoft reported. “The image suggests the bot had an internal struggle between knowing that boats float on lakes and the text specification of bus.”

Besides having fun just trying to confuse its drawing bot, Microsoft believes there could be practical applications for the technology. The company said it could be used as a sketch assistant for painters and interior decorators and, in the future, perhaps to create animated movies just by using the screenplay as its input source.

Image: Microsoft