Adam Fuchs, CTO of sqrrl, had a noteworthy chat with Dave Vellante and Jeff Kelly at the BigDataNYC 2013 event in New York, covering topics ranging from the Big data lessons learned at the NSA to the NoSQL database.

Because Apache Accumulo was initially developed within the NSA, everyone was expecting Fuchs to divulge some juicy government secrets.  Undoubtedly, interest for his presentation was quite high.

Monetizing + maintaining open source projects

Fuchs first commented on Accumulo’s shift from the NSA to open source, saying that “open sourcing software from the government and from the other side of the wall isn’t actually that hard. The hard part is continuing to participate in the project afterwards.”

As a start-up, the tricky part is deciding whether to do open source or closed source. “At some point you got to figure out how are you going to make money, but also how are you going to innovate and rally a larger development community than just your own team.”

Another point of interest seems to be the difference between Accumulo and HBase. “The two projects started around the same time but Accumulo was a closed source project inside the government for the first few years of its life. It’s an interesting discussion for multiple reasons, one being that the two projects have two different focuses, despite being both based on Google’s BigTable. Accumulo started focusing on the heavy ingest rates, and the HBase which had a focus on providing low latency query response times,” explained Fuchs.

At the moment sqrrl boasts having strong partnerships with Cloudera and Hortonworks, while also receiving a huge amount of funding through government contracts.

Lessons from the NSA

Vellante asked Fuchs to elaborate on the lessons learned at NSA and the security practices. The core axiom behind all these lessons learned is that “application development is the key for innovation in any organization,” believes Fuchs. “Application development is key for innovation in any organization.”

To Fuchs, applications are like the nuggets of innovation; the faster you can build those applications, the faster you can innovate. The very core of the Hadoop community is focused on avoiding the barriers to innovation.

Make sure you watch the entire interview below. On a not so distant note, Adam Fuchs’s Lessons in Big Data Security are explained more in-depth here.