UPDATED 09:03 EDT / APRIL 01 2015

Amazon’s new data-hungry D2 instances target Hadoop workloads

Cloud StorageAmazon has added a new kind of instance to its public cloud that is specially optimized to handle the massive amounts of data stored and processed in the distributed analytic clusters powering modern decision-making like Hadoop.

The batch processing framework splits work across multiple nodes that each hold a portion of the information to allow for rapid local execution of algorithms. The faster those algorithms can access the files, the quicker the results are generated and returned to the end-user, a process that the new instances aim to hurry along considerably.

The most expensive configuration in the freshly unveiled D2 series can retrieve up to 3.5 gigabytes of data  from the 24 two-terabyte disk drives attached to each virtual machine every second. That’s 40 percent faster that previous-generation HS1 instances it’s replacing.

Hadoop can take advantage of that speed boost using the 36 virtual cores included in the price, each of which corresponds to a thread on an Intel Xeon E5-2676 v3 processor running at a base frequency of 2.4 GHz. The 224 Gbits of memory allocated to each virtual machines on top of that provides plenty of room for the local algorithms to carry out the work, although that’s only enough for traditional implementations of Hadoop.

For more modern clusters that use an in-memory execution engines such as Storm instead of the default MapReduce paradigm, the best option is still the R3 series, which provides the same high-speed networking capabilities as D2 instances but uses previous generation Intel Xeon E5-2670 v2 chips. However, now that Amazon has turned its attention to Hadoop, it shouldn’t be long until the hardware is upgraded to the same standard.

Like its memory-optimized sibling, the D2 offers a wide range of pricing options to accommodate different classes of deployments. Regular instances range from $0.690 per hour for the cheapest 6-terabyte configurations to $5.520 for the ultra-fast d2.8xlarge, a rate that can drop over 80 percent with long-term usage commitments.


A message from John Furrier, co-founder of SiliconANGLE:

Support our open free content by sharing and engaging with our content and community.

Join theCUBE Alumni Trust Network

Where Technology Leaders Connect, Share Intelligence & Create Opportunities

11.4k+  
CUBE Alumni Network
C-level and Technical
Domain Experts
15M+ 
theCUBE
Viewers
Connect with 11,413+ industry leaders from our network of tech and business leaders forming a unique trusted network effect.

SiliconANGLE Media is a recognized leader in digital media innovation serving innovative audiences and brands, bringing together cutting-edge technology, influential content, strategic insights and real-time audience engagement. As the parent company of SiliconANGLE, theCUBE Network, theCUBE Research, CUBE365, theCUBE AI and theCUBE SuperStudios — such as those established in Silicon Valley and the New York Stock Exchange (NYSE) — SiliconANGLE Media operates at the intersection of media, technology, and AI. .

Founded by tech visionaries John Furrier and Dave Vellante, SiliconANGLE Media has built a powerful ecosystem of industry-leading digital media brands, with a reach of 15+ million elite tech professionals. The company’s new, proprietary theCUBE AI Video cloud is breaking ground in audience interaction, leveraging theCUBEai.com neural network to help technology companies make data-driven decisions and stay at the forefront of industry conversations.