Machine learning and deep learning have produced plenty of breakthroughs in recent years, from more capable speech and image recognition to self-driving cars. But one big problem with these artificial-intelligence techniques that attempt to mimic how the brain works is that the neural networks they employ don’t have the common-sense knowledge and context that people have, such as social conventions, laws of physics, and causes and effects.

That can makes their decisions sometimes perplexing or downright wrong — as anyone who uses Alexa, Google Assistant or any number of customer-assistant chatbots knows. But perhaps not for long.

Salesforce.com Inc.’s research team today announced a paper that outlines a way to improve that situation. In the paper, to be presented at the Association of Computational Linguistics‘ annual meeting July 29-Aug. 2 in Florence, Italy, the researchers propose training the neural network models not only with data from various data sets on how to answer many questions accurately but also with explanations for why those answers are the best.

Salesforce hired people on Amazon.com Inc.’s Mechanical Turk crowdsourcing service to provide those explanations for examples in the Commonsense Question Answering or CQA dataset proposed earlier this year by researchers at Tel-Aviv University and the Allen Institute for Artificial Intelligence. The result is a new data set Salesforce calls Common Sense Explanations or CoS-E, including examples such as these:

Image: Salesforce Research

“It’s basically able to take large amounts of unsupervised [unlabeled] text, distill common sense from that text, and provide reasoning for that,” Salesforce Chief Scientist Richard Socher told SiliconANGLE.

The surprising result, added Salesforce Research Scientist Nazneen Rajani, is that the neural network performed much better on the real test after seeing the examples of human reasoning only during the model training phase. “We speculate that the explanations capture valuable information about the way the world works and the network learns to reason based on that information at test time,” she wrote in a blog post.

In a second part of the research, the researchers, who also include Bryan McCann and Caiming Xiong at Salesforce Research, trained a second neural network only to learn how to generate commonsense reasoning from reams of text it has read to mimic CoS-E’s human-generated explanations. This Commonsense Auto-Generated Explanations or CAGE framework did better yet in terms of accuracy of answers, with a 65% score that beat out other recent methods — though it’s still well behind humans’ 95% accuracy.

McCann said he expects the results to get even better as the model becomes more familiar with knowledge of the world. He hopes the explanations will “fill in the gaps” in the neural network models over time.

Salesforce admitted in the paper that there’s another area of improvement needed, one that’s getting a lot more attention these days from labs to Washington D.C.: rooting out bias in the models because of bias in the data. For instance, in the CQA, Salesforce researchers noticed a “significant gender disparity and bias with higher proportion of female pronouns used in negative contexts.”

Rajani said it’s not yet clear how to deal with this bias, though it will probably need to happen in the model rather than the data, which is hard to clean of all bias. “We want to work on a knob for reducing bias,” McCann added, but at least the new work provides better explanations of how the neural network comes to this bias.

The net result of the research potentially could be less frustrating and more seamless chatbots and digital assistants, Rajani said.

Salesforce is planning to open-source a data set of commonsense explanations.

Image: johnhain/Pixabay