During the last day of the HP Vertica Big Data Conference in Boston, theCUBE co-hosts John Furrier and Dave Vellante had a very interesting talk with Chris Wegryzyn, Director of Data Architecture, Democratic National Committee, talking about the impact of Big Data in political campaigns.

Talking from his personal experience, Wegryzyn was able to provide valuable insight from his previous work on the Obama campaign, explaining the role Big Data and how analyzing large data sets contributed to winning the elections.

Vellante wanted to know how the data was used, and its impact. It turns out that the campaign manager wanted to measure everything, and he demanded analytics in every aspect related to the campaign. The challenge was to look where no one looked before, finding the pools of data that sat untapped and driving intelligence from them quickly.  It wasn’t a smooth and flawless process, as the team had the biggest challenge of all, “the mother of all deadlines” as Wegryzyn put it. There’s no point in delivering information after the elections.

Elaborating on the challenges faced then, Chris remembered how the team had to learn everything on site. They didn’t have any experience, the team was not familiar with SQL, there were no tech wizards. They were people who knew the dynamics or voting and fundraising, who were offered the ability to get into the data and “play around” with it. This was powered by Vertica, a vital component of the process that allowed to make this possible. The greatest thing about it was that the analysts were able to use the data right away.

The data sources were varied. There were public pools of data, registered voters lists, their voting habits, all the contributions coming in,  the volunteers’ interactions and the social data, to name but a few. There were also a lot of different paths to get the data into the platform, but they ended up being very unhappy with what they could find and finally built their own system.

The biggest data surprise of all

TheCube co-hosts were curious about the biggest surprise Wegryzyn had working for the campaign. He admitted it was the realization that it was actually going to work. The proof of concept was a sweet victory, as there were around 30, 40 people analyzing data at all times.

Asked about how he managed to balance work and the public persona with the personal life, Wegryzyn agreed that working on a political campaign is hard, and you have to really want to be involved, because of the hours it demands. Despite the hardships, they are recruiting great engineers and great analysts and they all agree the hardest part is lack of sleep.

However, the timelines make it tough, forcing a different way of looking at things. The constant dilemma was “What can we do in a week to add value?” The data was placed into the analysts’ hands and after a week they had to decide whether or not it was valuable data. If the answer was yes, they tried making it better. If the answer was no, they discarded it altogether.

Empower analysts

A lot of people working for the campaign have been involved for several years and there’s a sense of joint investment in this. “It’s always a great thing to have people excited about their job”, said Wegryzyn. He noted that there’s been a lot of growth in the social media area, but there’s still a lot to learn about the entire process.

His advice for the business folks is to “Empower analysts”. They are smart people who can acquire new skills and look at data in new ways, instead of being isolated in the organization.