The explosion of data volumes means there is more data available for analysis than ever before. But some types of data are more suitable for analysis than others, and storage technology needs to know the difference to ensure high performance.

In a recent Wikibon research note on the evolution of storage technology, my colleague David Floyer wrote:

Lastly but most importantly, storage will be a key enabler of new business process and business intelligence applications that will be able to digest and present orders of magnitude more data than current applications, an era of “Big Data”.

David further posited that solid state drives, or SSDs, will increasingly be used to store active data, while Serial ATA hard disk drives, or HDDs, will be used to store less active and inactive data.

SSDs store data on memory chips with rewritable cells and have no moving parts. I/O speeds are significantly greater reading data from SSDs than HDDs, which store data on spinning disks. SSDs are more expensive than HDDs, however.

Teradata’s new Active Data Warehouse 6680, released this week, is among the first analytic database platforms to employ an SSD/HDD hybrid model. The idea is that the platform stores data that is regularly used for analysis on SSDs so it can be accessed in near real-time. Data that is

not routinely analyzed – the bulk of the data in the warehouse — is stored on HDDs. It will take longer to read this data, but it at least makes it available when the need arises.

A key feature of the 6880 is Teradata’s virtual storage technology, or TVS, which analyzes the data to determine its ‘temperature.’ Teradata explains:

[TVS] continuously and automatically places the most frequently used “hot” data on the fastest, solid state storage and the least used “cold” data on the slowest storage without user or administrator intervention. Usage of data changes as it ages naturally creating dramatic changes to data temperature. TVS automatically moves data as temperature changes ensuring alignment to the most appropriate storage location. Teradata supplies a monitoring tool that helps customers know exactly how much of their data is hot so they can purchase exactly the right number of SSD’s for their needs.

Sales data from the last six months might be considered ‘hot’ because it is analyzed often to produce sales forecasts, for example. But as the data ages, it becomes less useful and turns ‘cold.’ TVS tracks data usage patterns to determine its temperature and stores it either on SSDs or HDDs accordingly. Automating this process is the key to a successful hybrid platform. Manually determining data temperatures and where to store it is unfeasible in the Big Data era. By intelligently combining SSDs and HHDs, platforms like the Teradata 6880 can be tuned for optimum performance

I expect we’ll be seeing more of the SSD/HDD hybrid model in the near future. I’d urge CIOs to talk to their data warehouse vendors about how they plan to tackle dual challenges of rising data volumes and increasing performance expectations from users. The SSD/HDD hybrid model has the potential to meet both challenges, though other options exist.