This Week in Big Data: From Infochimps to Twitter’s Flash DBs

This past week has seen a few interesting updates from the analytics space.  The first item on our list is the Infochimps Dashpot launch.

The dataset marketplace operator has been offering the Infochimps Platform to customers since earlier this year, and Dashpot represents a very large extension to the solution. The main focus is to make the data available to business users and not just data scientists: a topic that has been surfacing on a rather persistent basis lately, most recently in context of the EMC VSPEX launch which simplifies the infrastructure for easier implementation.

Infochimps is taking a different approach though.  Its secret recipe includes several doses of real-time visualization, as well as quite a few other additions that round out what Dashpot’s data-driven capabilities.

Next up is the Cloud 9IDE update. As the name implies, the startup offers a hosted development environment for developers, which is also open source and supports about two dozen different languages. One of those is Node.js, and as of this week Node apps can be deployed directly on VMware’s Cloud Foundry via the Cloud 9IDE interface.  The language can be particularly handy  when coding big data applications.

Last but not least is the Twitter news. The social network, which generates practically all of its revenue from selling real-time data to marketers, open-sourced some of its MySQL tricks. These were developments to help run the site better although, as Twitter itself clearly noticed, they could be used in many other environments.  Some of the modifications to the language include optimization for SSD hardware, more variables and other techniques meant to keep the Twitter fail whale well exercised.