If you were watching television (or even alive) in the late 1970s, you would be familiar with a rather brilliant advertising campaign by Memorex, touting the quality of its recording tapes by comparing the sound coming from a TV and the real thing. The most memorable had the inimitable Ella Fitzgerald hitting a high note and breaking a wine glass, only to have the camera pan back as she was watching herself on TV with the broken glass next to her.

One very prominent artificial intelligence researcher and A.M. Turing Award winner Judea Pearl argues in his latest book (with Dana Mackenzie), “The Book of Why: The New Science of Cause and Effect,” that AI has been handicapped by an incomplete understanding of what intelligence really is.

I tend to agree with this. In a recent article on LinkedIn, I wrote, “If you think about it, each new step towards AI, rather than producing something which everyone agrees is real intelligence, merely reveals what real intelligence is not.” What I meant by that was that, before we had facial recognition software of conversational natural language processing, we assumed that was AI, but it turned out it wasn’t really intelligence, or reasoning. It was just artifice.

The big wow factor in the recent AI spring — AI winter is generally referred to as the period between the first attempt at AI the 70’s and 80’s, and the ensuing period until recently — is an explosion of interest and investment in machine learning and AI. There were the flashy parlor tricks such as IBM Corp.’s Watson beating the best Jeopardy champion (though very little of the “intelligence” of that Watson machine was preserved for the later production versions), Apple’s Siri or AlphaGo, the Go-playing machine built by researchers at Google LLC’s DeepMind lab beating Chinese grandmaster Ke Jie.

Pearl is unimpressed by these things. As a pure contrarian, he believes that none of this is new, just Moore’s Law-enhanced computation of disciplines that existed decades ago: swimming through lots of data looking for things. What he said really got everyone’s attention:

“All the impressive achievements of deep learning amount to just curve fitting.”

Just curve fitting. That’s pretty harsh, but is it valid?

Curve fitting: According to Wolfram Research’s curve fitting documentation, “There are many situations where one wants to find a formula that best fits a given set of data.” That sounds simple enough, except it’s not. There are 10 to 20 standard algorithms, from simple linear regression to adversarial neural networks, and any good experimenter applies them after making many decisions about applying them, from selecting test data to choosing features.

Some machine learning algorithms that are common in use are linear regression, logistic regression, linear discriminant analysis, classification and regression trees, Naive Bayes, k-nearest neighbors, learning vector quantization, support vector machines, bagging and random forest, and boosting and AdaBoost.

It should be noted that every one of these was developed before the onset of big data and machine learning. What characterizes investigations of using these algorithms is the vast amount of data they now consume and the vast amount of computing resources.

Just curve fitting? When you consider just how sophisticated it can be, Pearl’s use of “just” has to be taken in context. Pearl believes that to move forward with AI, we have to replace “associative” reasoning with “causal” reasoning. His contention is that, not matter how huge your datasets, no matter how many computation cycles you have, if your goal is to tease out associations and patterns in the data for prediction, you are not doing AI. His vision is how machines will be able to think, and what is lacking in ML today is “causal” reasoning.

On a side note, I had the privilege to work with Pearl 20 years ago where I first learned about his invention, Bayesian Belief Networks. If I had to sum it up, I’d describe the difference between BBNs and ML this way: The former starts with something you already know, while ML assumes you know nothing, except for the intended or inadvertent bias you introduce into the model.

Pearl does not dismiss the value of ML. There are clearly excellent examples where it is extremely useful. But, he says, “as much as I look into what’s being done with deep learning, I see they’re all stuck there on the level of associations. Curve fitting. That sounds like sacrilege, to say that all the impressive achievements of deep learning amount to just fitting a curve to data. From the point of view of the mathematical hierarchy, no matter how skillfully you manipulate the data and what you read into the data when you manipulate it, it’s still a curve fitting exercise, albeit complex and nontrivial.”

“Albeit complex and nontrivial.” Instead of conflating ML with AI, I adopt the definition of nonsentient machine intelligence, typically focused on a narrow task. And those narrow tasks can be extremely useful. The real magic in applying machine learning models to a software product is producing the right mix of things that are general enough to work with a wide range of situations and powerful enough to produce nontrivial results repeatedly, not useless examples such as: “Most auto injury accidents occur when the driver is at least 16 years old.”

Pearl’s new book is available now, and there’s further academic reading on his premise in this paper.

My take on all this is Pearl is looking ahead to true AI, or what we refer to as Artificial General Intelligence or AGI. Without the ability to reason and understand cause and effect, it’s just, well, curve fitting.

Image: ThatEricAlper.com