Google LLC wants to make machine learning models more transparent in order to protect against bias in their decision-making.

Bias is a big problem in artificial intelligence, of which machine learning is a subset. It’s a byproduct of the black-box nature of AI applications, which don’t always reveal the logic behind their decisions.

Bias can be introduced in training data as well as machine learning logic, and teasing out such aberrations can be an arduous data science exercise. As a result, many enterprises are still reluctant to trust AI when it comes to making important decisions.

Google Chief Executive Officer Sundar Pichai said today at the company’s annual I/O conference for developers that transparency matters to the company because many of its technologies, such as Google Assistant, increasingly rely on AI to carry out tasks.

To that end, the company is pursuing a new research approach called testing with concept activation vectors, or TCAV, which it describes as an “interpretability method” to help understand the signals neural networks use to come to their decisions.

What TCAV does is it tries to dissect machine learning models in order to try and understand why they make certain decisions. For example, Pichai said, “TCAV could reveal if a model trained to detect images of ‘doctors’ mistakenly assumed that being male was an important characteristic of being a doctor because there were more images of male doctors in the training data.”

TCAV can surface other issues that could potentially lead to bias too, such as race, income and location, Pichai said.

Google is open-sourcing TCAV so others can use the technology to make their machine learning models more transparent, and will be releasing more tools and open data sets “soon.”

The open-sourcing of TCAV was welcomed by Holger Mueller, principal analyst and vice president of Constellation Research Inc., who said that although AI is a fundamental enabler of future enterprise software, valid concerns remain over its exact capabilities. Mueller said that’s especially true with neural networks, which have mostly evolved beyond the point of explaining why they make a decision.

“That opens enterprises to potential liability issues and overall bad business, so it’s key for AI vendors to find ways to mitigate this risk,” Mueller said. “Today it is Google’s turn using TCAV as a valid methodology to reduce AI operational risk.”

Google isn’t alone in pushing for greater transparency in machine learning. Recently, IBM Corp. announced its AI OpenScale platform that logs every prediction, model version and all training data in order to help organizations understand how AI applications reach decisions through plain-English explanations.

Photo: Joseph Davidson/Flickr