Object storage system provider Cloudian Inc. and deep learning company Skymind Inc. are teaming up to provide data management solutions for artificial intelligence systems.

Emerging technologies such as AI and machine learning are seeing rapid adoption as companies bet on their potential to tackle problems that simply couldn’t be solved before. But Cloudian and Skymind say AI and machine learning present their own challenges, as the massive data sets needed to train these systems become increasingly difficult to manage.

To tackle this, Cloudian said, it’s joining with Skymind to offer a number of integrated and enterprise-ready AI and machine learning solutions based on its object storage platform. The plan is to integrate Cloudian’s HyperStore object storage systems for on-premises deployments with Skymind’s Deeplearning4j framework, which is used to train deep learning models and provide actionable intelligence from the data it crunches.

The Deeplearning4j framework is an alternative to Google Inc.’s better-known TensorFlow deep learning library. Both are based on the Java programming language, but Skymind said its offering is better-suited to enterprises because it comes with commercial support that’s not available with TensorFlow. In addition, the Deeplearning4j framework supports big-data technologies such as Apache Hadoop and Apache Spark, and it’s optimized for Intel Corp.’s newest-generation Xeon Phi server processors, IBM Corp.’s POWER series processors and Nvidia Corp.’s graphic processing units. Skymind claimed that this improves hardware utilization and therefore makes it possible to train artificial intelligence algorithms faster.

The new partnership centers around training data for AI and machine learning systems, the companies said. Basically, Cloudian provides the hardware to extract and store this data, before making it accessible to the Deeplearning4j framework, which trains the actual systems.

“AI and machine learning presents two storage challenges,” said Michael Tso, co-founder and chief executive officer of Cloudian. “First, you need lots of capacity, because to have effective pattern recognition, you need a lot of training data to refine your detection algorithm.”

The second problem is that this training data needs to be “tagged: in order to improve the accuracy of machine learning algorithms.

Tso cited an example use case last year, when Cloudian demonstrated its object storage platform’s suitability for AI workloads, deploying its systems inside advertising billboards in Japan in order to deliver targeted advertising to individual drivers based on the likely preferences of each car owner’s demographic.

“If you have a car picture – a BMW in the rain at night, for example – you’ll want to tag the picture with the car type, weather conditions and time of day to improve your matching accuracy,” Tso explained. “Cloudian stores user-defined tags with the data to make that record keeping easy.”

The companies said their combined technologies should prove especially useful to industries such as healthcare, financial services, security and robotics.

They’re also touting a new security monitoring tool that they say is the first of several integrated AI- and machine learning-based solutions they plan to offer. The new tool is designed to identify attacks on corporate networks, using pattern recognition to identify suspicious activity. In this case, Cloudian’s HyperStore object storage systems take care of the data management, which in turn supports the training environment, algorithm development and ongoing refinement that’s handled by Deeplearning4j.

Screenshot of Cloudian and Skymind’s new network intrusion detection tool

“Deep learning can produce state-of-the-art accuracy on many problems, and it requires large datasets to do that,” said Skymind CEO Chris Nicholson. “That’s why it makes sense for a deep-learning company like Skymind to partner with a hyperscale storage provider like Cloudian. These are two crucial parts of a larger solution.”

Image: A Health Blog/Flickr