If you’ve seen any indication that humans are getting smarter and more sophisticated, please inform me, and you don’t have to read what follows. For everyone else, who sees no lack of stupidity and misinformation in both business and public life, read on about the joys big data will bring.

But, first, a quick pass down the vitamin aisle at the local Whole Paycheck. Over the past 25 or so years, I’ve taken megadoses, added fish oil, added CQ-10, subtracted megadoses, added smaller doses, switched to a one-a-day, all basing my decisions on what seemed like the best available data at the time.

It’s a bit like cardio-pulmonary resuscitation, where the American Heart Association had us doing it very much wrong for decades, at least if you believe that current and drastically different CPR protocols are saving lives that would have been previously lost. Which they are.

Big medicine and big pharma already use lots of relatively big data and have since always. Well, almost that long. Yet, the answers they come up with seem to double back on themselves with amazing regularity. What’s a guy supposed to believe?

And that’s where big data arrives. Whatever big data your company gathers, about customers, suppliers, transactions and such, ought to reveal the image of God. Buried in the data is the sum total of your corporate experience, all reduced to numbers.

When thinking of Big Data, I sometimes also think about the warning, often improperly attributed to Mark Twain, that “figures don’t lie, but liars figure” and to my general belief that most people lack the consistent ability to analyze the information presented to them and take the proper action.

Many people are already being led around by those who manipulate the information they receive – sometimes they even do it themselves. American’s recent rejection of facts for comfortable opinion and pseudoscience should scare us all.

Companies may be smarter in how they handle data analysis, but that’s a big maybe. End-users have no monopoly on logic bombs and it is, after all, end users who make decisions for businesses, too.

The insights offered by analysis of big data are only as good as the human beings that create the data, gather and assemble it, decide what questions should be asked and how the data is presented. And interpreted, especially that.

I wrote a piece recently about PowerBI, a “business intelligence for the masses” that works with Microsoft Excel and supports natural language queries, charts, maps, pivot tables and other tools in search of the holy grail of data turned into intelligence.

When Bad Answers Happen from Good Data

.

Time will tell whether average users can make PowerBI work effectively without some training in how to tease meaningful – and actionable – information from a mess of data. And how often the data will be there, but the humans – even with their Microsoft helper app – won’t understand what they should be seeing and will conjure up a bad answer from good data.

There was a time, back when IBM ruled the cosmos, that many felt any answer a computer churned out had to be right. As we folded, spindled and mutilated our way through the 1960’s, we realized computers could only figure out as much as the people who programmed them.

I think we may be headed into another period of brain fade, in which the sheer amount of data we are able to manipulate – never say “command” – and the excellent tools that generate what amount to instant infographics (passing for actionable information) will get us into trouble.

The answers may seem so complex, especially to us non-data-scientists that extra doses of business buffoonery will arrive and be accepted as fact because some big data genius churned them out.

We badly need to train people in critical thinking, probability/statistics, and the other skills necessary to lines of business and IT folks to understand what big data is trying to tell them – and what it is not.

I’d like to see someone start a really big training company to help end-users and lines-of-business folks learn how to better create useful data in the first place and then understand what the data can tell them. I realize this could be a four-year degree program, but we need an Edward Tufte, perhaps, to help ordinary mortals explain big data.

I am very concerned that big data, misapplied and misunderstood, will create big lies. Or more likely a combination of lies and truths that prove very difficult to sort out.

It is up to you, dear reader, to help keep this from happening. To gather the right data, interpret it well, understand its meaning, and create plans that are understandable, accessible and actionable.

Big data doesn’t lie, but people can make it look that way.

photo credit: ChrisyJewell via photopin cc