UPDATED 17:56 EDT / FEBRUARY 02 2017


Facebook’s AI lets you search pictures by what’s in them, no tags needed

If like most people you don’t take the time to tag and name all of your photos, then you know that searching for a specific picture can be a nightmare. Thanks to Facebook Inc. and artificial intelligence, that problem may soon be a thing of the past.

Facebook has been investing heavily in artificial intelligence research over the last few years, especially when it comes to computer vision. Today the social network announced an update that will allow users to search for images by content rather than by their name or tags.

“In other words, in a search for ‘black shirt photo,’ the system can ‘see’ whether there is a black shirt in the photo and search based on that, even if the photo wasn’t tagged with that information,” explained Joaquin Quiñonero Candela, director of engineering at Facebook.

The new feature takes advantage of Facebook’s Lumos AI, which uses computer vision to recognize people, animals and a number of other objects in a photo. Lumos can even take things step further by recognizing what is actually happening in a photo, so in addition to recognizing that a person is in the image, Lumos could also determine whether that person is walking, riding a horse, playing an instrument and so on.

According to Candela, the technology developed for Lumos has a wide range of applications beyond simply improving image searches.

“More than 200 visual models have been trained and deployed on Lumos by dozens of teams, for purposes such as objectionable-content detection, spam fighting, and automatic image captioning,” Candela explained. “The applications are wide-reaching, with everyone from our Connectivity Labs to Search to the Accessibility team using the technology.”

Facebook built Lumos using its FBLearner Flow platform, a general-purpose machine learning pipeline that the company created so that its engineers could implement multiple AI projects without having to provision the necessary hardware each time. According to Facebook, there are currently more than 1.2 million AI experiments running on FBLearner Flow each month, and all of these projects, including Lumos, are able to access the never ending stream of data coming out of the social network and its many apps.

Although Facebook is making some impressive progress with Lumos and similar projects, the social network is not the only tech giant working on computer vision. Alphabet Inc. has been developing its own picture-scanning AI to boost image searches, and the company boasted last year that its Inception v3 AI could caption the content of an image with 93.9 percent accuracy. Inception v3 made headlines again recently as the basis for a new AI designed to help doctors catch skin cancer early.

Image courtesy of Facebook

A message from John Furrier, co-founder of SiliconANGLE:

Show your support for our mission by joining our Cube Club and Cube Event Community of experts. Join the community that includes Amazon Web Services and Amazon.com CEO Andy Jassy, Dell Technologies founder and CEO Michael Dell, Intel CEO Pat Gelsinger and many more luminaries and experts.

Join Our Community 

Click here to join the free and open Startup Showcase event.

“TheCUBE is part of re:Invent, you know, you guys really are a part of the event and we really appreciate your coming here and I know people appreciate the content you create as well” – Andy Jassy

We really want to hear from you, and we’re looking forward to seeing you at the event and in theCUBE Club.

Click here to join the free and open Startup Showcase event.