Data combined with insight can change the world for the better, Tableau CEO and Cofounder Christian Chabot told a rapt audience of 8,300 users and senior IT executives at the Tableau User Conference. He started his speech on how to change the world with a simple, hand written spreadsheet tracking deaths in a hospital in Vienna in the 1860s. This, he said, was the data that Dr. Ignaz Sammelwiez used to develop the germ theory of disease and the basic insight that doctors were spreading disease by failing to wash their hands between patients, an insight that has saved millions of lives.

He showed a star chart to discuss how Astronomer Henrietta Leavitt discovered that 25 stars provided the key to determining the distance from earth to any star, a basic discovery that led to an understanding of the size of the Universe and eventually to such advanced insights as dark energy. These and other world-changing discoveries were made by individuals who developed revolutionary insights based on data.

Four Components of Insight

So how did these leading thinkers develop their insights? Chabot identified four major components of insight that differentiate brilliance from the ordinary:

The first of those is intuition. An expert can develop a feeling for the data and how things work that goes beyond common understanding. Albert Einstein developed that kind of informed intuition, and it led him to develop General Relativity and his famous equation.

The second is that they look for the unexpected, the outliers that provide something significant. To illustrate this he retold the story of Scottish microbiologist and botanist Alexander Fleming who, he said, always had a messy workbench and made fun of his co-researchers who kept their spaces completely ordered. One day in 1928 he returned to work to find that something unexpected was happening in a petri dish he had left open overnight. A mold had invaded the bacterial culture in the dish and had created a dead space around it. That mold was Penicillium notatum and the substance it used to kill the bacteria was penicillin, which saved thousands of lives in WW II and started the modern age of antibiotics, and earned him a share in the 1945 Nobel Prize in Physiology or Medicine.

The third element of insight is shifting perspectives. To illustrate that, Chabot displayed the four phases of Dmiti Mendelayev’s development of the Periodic Table of the Elements, starting with an alphabetical listing. Although Chabot didn’t mention it, this is also a demonstration of the power of visual representation of data. Mendelayev’s Periodic Table, which showed multidimensional relationships among different elements such as the noble gases, has been used to predict new elements and is the basis of modern chemistry.

Finally, he said, insight is based on the ability to relate disparate pieces of data on the fly. Both Issac Newton and Charles Darwin had this ability, which underwrote their basic insights into the nature of reality.

To illustrate how genius uses these four elements to develop major insights, he told the story of Archeologist David Steward and the Mayan Code. The Mayans carved the record of their civilization into the walls of their great temples and palaces using a complex pictographic code. The problem is it has an inconvenient number of glyths — about 500 — too many to be an alphabet but far too few to be a hieroglyphic representation of words like ancient Egyptian or modern Chinese. For generations archeologists had puzzled over this to the point that some had suggested that the figures were not writing at all but mere decoration. Steward’s insight was that the Mayans used more than one pictoglyph to represent the same letter. They were being creative in their writing. That basic insight allowed Steward to translate Mayan writing and create a window into ancient Mayan civilization.

Empowering Analysts with the Keys to Insight

These four elements are the keys to turning data into insight, Chabot said, and they can be used by business and financial analysts as well as great thinkers. Unfortunately, he said, traditional data analysis packages are designed to work in a way antithetical to the process of human insight. They are rigid, restrictive, and do not allow analysts the freedom to experiment, follow their intuition, pivot to follow where outliers in the data may lead, or combine disparate data to see what might be discovered. They do not support intuition.

Tableau, he said, is dedicated to developing the tool that analysts need to turn analysis into insight. That means a tool that is highly flexible and allows users to combine the data with intuition to discover new insights rather than just turning out the same old reports that support old ways of thought. The new features in the upcoming versions of Tableau, some of which were announced in the second half of the general session, were inspired by that dedication combined with the requests of Tableau’s users.

Chabot repeated some of these ideas in his interview on TheCUBE from the Tableau conference.