Yahoo just made deep learning easier with CaffeOnSpark

Yahoo just made deep learning easier with CaffeOnSpark

Yahoo! Inc., is getting into the artificial intelligence (AI) game with the release of new internally-built software under an open-source license. Called CaffeOnSpark, the software is able to perform ‘deep learning’ on the vast ocean of data kept in Yahoo’s Hadoop file system. Now, the company has made it available on GitHub for everyone to use.

Deep learning is a machine learning method that’s particularly useful in helping computers come to sort through and recognize user-generated data, and one of its most exciting use cases is where images are concerned. As such, Yahoo built CaffeOnSpark to help identify the billions of images posted onto its Flickr photo sharing website.

Yahoo explained that the idea behind CaffeOnSpark was to make the search function on Flickr more useful, so that it could pick out suitable images based on more than just the descriptions and tags users write when uploading their pics. CaffeOnSpark is able to learn and recognize the specific common characteristics of certain kinds of objects, animals and landscapes, for example.

Perhaps the most surprising thing is that Yahoo came up with this technology all by itself. Yahoo isn’t exactly considered to be at the cutting edge of tech anymore, but the release of CaffeOnSpark suggests it’s not so far behind bigger names like Google and Microsoft in the AI stakes at least. Yahoo’s move comes just a few months after Google outsourced its TensorFlow machine learning framework, a move that was quickly followed by Microsoft, which open-sourced its own CNTK machine learning framework just a few weeks later. Other companies, including Facebook and China’s Baidu, have also open-sourced their own machine learning technologies in recent weeks.

Yahoo’s take on machine learning is interesting though. One of the main disadvantages of deep learning is you need to move around massive amounts of data. For example, if you want to teach a search engine how to recognize dogs in photographs, you need to feed it a huge number of canine images so it knows what to look for. This usually involves transferring all of those images to new servers, which is costly and time consuming.

RELATED:  What you missed in Big Data: The AI arms race continues

However, Yahoo did things differently. As the name suggests, CaffeOnSpark is a combination of two technologies – the Caffe framework, and the Apache Spark data processing engine, which sits atop of Hadoop. In other words, Yahoo has created a way to perform deep learning on existing databases without any need to move that data around.

It seems unlikely that Yahoo is going to emerge as a leader in the field of machine learning, but its innovation will certainly benefit those who are at the forefront of this exciting new technology.

Photo Credit: aftab. via Compfight cc

Mike Wheatley

Mike Wheatley is a senior staff writer at SiliconANGLE. He loves to write about Big Data and the Internet of Things, and explore how these technologies are evolving and helping businesses to become more agile.

Before joining SiliconANGLE, Mike was an editor at Argophilia Travel News, an occassional contributer to The Epoch Times, and has also dabbled in SEO and social media marketing. He usually bases himself in Bangkok, Thailand, though he can often be found roaming through the jungles or chilling on a beach.

Got a news story or tip? Email Mike@SiliconANGLE.com.

SIGN UP FOR THE SiliconANGLE NEWSLETTER!

Join our mailing list to receive the latest news and updates from our team.

SIGN UP FOR THE SiliconANGLE NEWSLETTER!

Join our mailing list to receive the latest news and updates from our team.

Submit a Comment

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

Share This

Share This

Share this post with your friends!