As the Spark Summit 2016 event continues in San Francisco this week, attendees are getting to learn about the latest uses for Spark in the tech world, as well as getting exclusive glimpses of its effect on future developments.

Christopher Cuong T. Nguyen, cofounder and CEO of Arimo, Inc., joined cohosts John Walls and George Gilbert (@ggilbert41) of theCUBE, from the SiliconANGLE Media team, during Spark Summit to discuss what his company is doing with Spark to create more effective and powerful work environments, along with the importance of keeping things accessible for all users.

Information and interactions

Nguyen, who described himself as “a very early adopter of Spark,” was confident in his company’s investment in Spark, saying, “If you are familiar with technology evolution, and then you understand architecture, and you have a sense of timing, then [an investment in Spark is] actually not a very risky bet.”

He also provided a deeper look at their motivations for using Spark specifically. “The information asymmetry that we had looking at Spark is that we looked at a whole bunch of different compute architectures, specifically in memory, and … we knew that what had to happen is that you need to have what’s called a distributed dataset that exists outside of the compute cycle,” he said.

Nguyen added, “Spark is unique in that it has the concept of RDDs [Resilient Distributed Dataset], and RDDs exist whether there’s compute cycle or not, and what we wanted to do was to build an interactive application on top of such a compute system, and you can’t do that without having the distributed dataset. So that’s one of the key things about the Spark architecture that we knew was necessary.”

He continued to explain that the other key was timing and referred to Moore’s Law in finding a viable point between evolution and affordability with memory.

Easy tuning

Nguyen continued by exploring some of Arimo’s goals and ways of making its results available to customers.

“For the most part, when a data-scientist [adjusts hyper-parameters of algorithms], it’s pretty much a manual process; it’s a lot of guess-work based on experience,” he explained. “And so, it turns out that takes a lot of time. The training itself is run by large-scale computing … but the choice of parameters … that, if there’s a way to automate it, then that could save a lot of time and effort.”

Arimo’s overarching goal is “intelligent augmentation … and what that means is that we augment human intelligence with machine intelligence in the enterprise.” Nguyen said.

Worth for customers

Nguyen’s next focus was to explicate the “two major human targets” of Arimo’s product suite. “One is the business user, the other is the data scientist. … For the business user, we allow the business user to go to a web interface, a document (we call it a narrative), and type in a natural-language question … and have the answer come back within 10 seconds,” he explained. “In order to answer that question, you need a lot of predictive models and the computing power and the data processing required to do that.”

He continued, “But [for] the data-scientist … we provide ways to make the data centers a lot more productive, and we also work on deep-learning algorithms so that there are new ways that the data scientist can use, for example, time-series processing. The third value proposition that we give to the business user and the data scientist is the collaboration between these two. … The value to the enterprise is expressed, or is most leveraged, when these two have a common environment and express their unique skills.”

Describing the current structuring of question-answering software, particularly in regards to how it verifys input and identities as being what they truly are, Nguyen characterized it as “dumb” because “there’s no learning behind it.” What Arimo is aiming for is to change that by applying machine learning to machine learning itself.

Watch the full interview below, and be sure to check out more of SiliconANGLE and theCUBE’s coverage of Spark Summit 2016.

Photo by SiliconANGLE