The Kinect tagline “You are the controller,” just took another turn for the different when a homebrew use of the device showed how the technology could recognize buttons drawn onto a piece of paper. This is the outcome of a contest run by Willow Garage—best known for their open-source Robot Operating System. The first place winner happened to be a “drawable buttons” hack to the Microsoft Kinect peripheral.

The best article floating around the Internet comes from the Popular Science blog,

This ROS contest was designed to find the most creative and useful hacks the Kinect community could come up with, and Garratt Gallagher’s customizable buttons hack won handily.

The interface requires a white or just barely off-white paper and a black or very dark pen that can draw in thick strokes. It uses ROS and the OpenNI Kinect driver, and seems like a fairly simple hack, as far as these things go. (Willow Garage is nice enough to provide instructions and demos on their contest page.) It uses the Simple Fast Media Library for sounds, though it’s not unreasonable to think other versions could be made that use just about any sound you want.

The implications of this particular homebrew code extend a lot further than what’s currently being shown. As anyone who has ever played Okami or Scribblenauts might point out about how powerful “drawing” in video games happens to be. It has some far-reaching fun-factor elements when it comes to interacting with video games, for sure, such as being able to draw your own controller, or modify one already delivered to better suit your play style.

The real innovation I see here, though, happens to be the optical recognition capability of the Kinect and therefore the ability to deliver specialized controllers (or even a book of them.) Since the system can recognize them from a piece of paper, we could have controllers that are basically squares of cardboard with elements on them that the Kinect can latch onto—enabling all sorts of variation in both game play and interaction. (The only problem here is that it would be mostly useful to prop-based gimmick games and everyone hates those. Props are so easy to lose.)

Or, we might just see the advent of further augmented reality products that the Kinect and DLC content could recognize. Already we’ve seen this with baseball cards and PC webcams, the Kinect is only a step away from this sort of content. I already see a market for Yu-Gi-Oh video games taking advantage of something like this. Episodic content could be released online for thousands of crazed American animé fans (most of whom are pre-teens) who want to actually see the Burger Time Ritual take place on their TV set when they pull out the card, cackling and screaming as they game against their friends. The stuff of cartoons is now actually in our hands.

This sort of optical recognition also opens up the availability of coupons, DLC links, and other things that could be printed into traditional media for use with the Kinect. This already is in effect with smartphones which can scan in QR codes just by photographing or “seeing” them. Add segments like this to collectable card games, print advertising, other products—perhaps paint it on with a pigment that cannot be seen by human eyes but the Kinect can see fine—and then bring it into view and viola, the user suddenly has some sort of value-added effect that emerges from their already working product.

What about clothing? Specifically branded t-shirts could become a sort of gimmick for game and media producers to recognize fans—and give them a reason to buy their swag—either giving them some sort of in-game benefit, social-media avatar effect, or whatever else advertisers want to do with it. Especially noting the advent of Avatar Kinect, marketers would love to know metrics on who chose to wear their product in front of the camera.

The innovations and properties of this sort of detection are endless when it comes to social and marketing implications, but also extra avenues for interacting with devices that have been granted as sort of “sight.”