IBM Corp. said today it has made some big improvements to the natural language processing capabilities of its IBM Watson platform.

The new capabilities, which were born out of IBM Research’s Project Debater, will help Watson understand and analyze some of the most challenging aspects of English language with greater clarity than before, the company said.

IBM’s Project Debater is an artificial intelligence system that was built by the company to debate with humans on a complex range of topics. The system is designed to understand subtleties in the English language such as idioms and colloquialisms that traditional AI has always struggled with.

For example, a phrase such as “hot under the collar” is well-understood by humans, but most AI systems are likely to get the wrong end of the stick, so to speak, since their algorithms can’t detect the true meaning. But those kinds of idioms aren’t a problem for Project Debater, which is capable of much more advanced sentiment analysis.

IBM said it will be integrating Project Debater’s natural language processing capabilities into Watson “throughout the year.” Customers will then be able to add the new capabilities to their existing AI models to help them better exploit natural language.

“Language is a tool for expressing thought and opinion, as much as it is a tool for information,” Rob Thomas, general manager of IBM Data and AI, said in a statement. “This is why we believe that advancing our ability to capture, analyze and understand more from language with NLP will help transform how businesses utilize their intellectual capital that is codified in data.”

IBM said Project Debater adds four distinct capabilities to Watson, including more advanced sentiment analysis. As a result, it said, it can better identify and understand complicated word schemes like idioms and so-called “sentiment shifters” — combinations of words that, taken together, take on new meaning, such as “hardly helpful.”

Watson will also gain the ability to pull textual data from multiple sources and provide users with a summary of what’s said or written about a specific topic. There’s also a new “clustering” feature that will enable Watson to cluster incoming data to create meaningful topics of related information that can later be analyzed. In turn, this will allow customers to customize each topic to reflect the language of specific industries and businesses, such as insurance, healthcare or manufacturing.

Finally, Watson will gain the ability to classify clauses that occur in business documents such as contracts from as few as several hundred samples.

IBM said the advanced sentiment analysis capabilities will be added to Watson next month, with the rest of the features to be rolled out throughout the course of the year.

Here’s a demo of Project Debater’s capabilities from IBM’s Think 2019 event in San Francisco last year:

Photo: IBM Espana/Flickr