People charged with making sense of big data should understand one thing: You’re working against human evolution. Not surprisingly, that can cause problems.

“Big data is very sexy and makes it seem like we can figure out anything. Well, maybe,” says Dr. Michael Shermer, publisher of Skeptic magazine. Shermer, who holds a Ph.D. in the history of science, has written more than a dozen books, including “Why People Believe Weird Things,” that seek to explain the logical frailties of human beings.

Shermer says humans are wired for a “right now” view of the world, strongly influenced by emotion and prior experience. When people don’t understand, they make things try to fit into the world they already think they understand.

“That’s why things like evolution and climate change are so difficult for most of us to grasp intuitively. Your ideology can take over what you think about them,” Shermer says.

People confuse weather with climate, for example, because “our brains are wired to care about weather, what’s happening now, not what is happening over the next 1,000 years. In our evolution, it has never been necessary to understand something happening that far out to survive.”

Related to understanding meanings hidden in big data, Shermer says people can make mistakes simply because humans aren’t wired to process information in huge quantities or think in the ways business or science often require.

5 common errors

 .

“The biggest problems are the cognitive biases that affect us when we try to sort through information,” Shermer said, outlining four of the most common causes of errors.

They are:

1. Confirmation bias — This is the tendency to look for and find confirming evidence for what you already believe and ignoring the disconfirming evidence.

“You selectively pick out anecdotes or stories or data that seem to fit with what you are already looking for. You can have statistical programs that hopefully weed out your bias in gathering the information, but bias then shows up buried in what questions you are asking and what sort of searches you do.”

“But it’s even worse than that,” Shermer says. “You end up thinking you are being unbiased and that the facts speak for themselves, but, of course, they don’t.”

2. Hindsight bias — This is where people know what has already happened and go in search of causes and explanations for it. “That can lead you to reinterpret information to make it seem to fit what took place. That can be misleading,” Shermer says.

3. Correlation vs. Causation — Just because two observation seem to fit together, such as “everyone drinks water” and “everyone who breaks their arm drinks water” does not mean that drinking water causes broken arms. Those facts happen to line up parallel to one another, but that is different from being connected.

“This is error #1 for beginning psychology students,” Shermer says. “It’s not that one thing causes another, it’s that people who do one thing are also likely to do the second thing.”

4. Too Much Data — “It’s hard to do a reality check when you are buried in the data,” Shermer says. “The problem is there’s so much data, so massive, that it would be surprising if you found anything that wasn’t significant. You end up with a deluge of things you could study, but which of them really matter?”

5. The Wrong “Big Question” — Not surprisingly, asking the wrong question up front can lead to what are mildly called “disappointing results.” Making sure the answer to the question actually resides in the data is the all-important first step in making big data make sense for business.

“When you have your head buried in the data set, it’s good to pull out sometimes and do a reality check,” Shermer advises.

Not surprisingly, in harnessing big data, as in the rest of life, Shermer says it pays to be a skeptic, always considering other possible causes for the answers you discover.

“Big data can be very useful, but just like anything else it needs to be taken with a grain of ‘skeptical salt,'” he concludes.

photo credit: JohnGoode via photopin cc