Cloud service providers (CSPs) such as IBM SoftLayer, Microsoft Azure and Rackspace Inc. rank high on the list in terms of virtual machine (VM) performance. But factor in price, and a different picture emerges.

When overall price-performance is considered, 1&1 Internet Inc.’s Cloud Server is at the top of the list, according to a new report from Cloud Spectator, “2016 Cloud Vendor Benchmark Report, North American Report” (available for free using coupon code 2016REPORTS).

Using Cloud Spectator’s methodology, 1&1 ranked highest using Cloud Spectator’s CloudSpecs Score. This is due in large part to high virtual machine (VM) performance and the company having the least expensive packaged pricing compared to other providers at the time of the study.

Higher costs affected the ranking of providers such as IBM SoftLayer and Microsoft Azure. They may have above-average performance for VM environments, but higher costs brought down their overall price-performance value.

Amazon Web Services (AWS)—which scored higher in VM performance than CloudSigma Holding AG, ProfitBricks Inc., Google Compute Engine and Phoenix NAP LLC —did not make it into Cloud Spectator’s Top 10 because of lower relative disk performance and higher monthly costs.

1&1 received the highest CloudSpecs Score for overall price-performance and block storage value.

It’s important to evaluate price-performance value because selecting a CSP is not black and white, said Kenny Li, CEO of Cloud Spectator.

“In most cases, you’re never just looking for performance. And in most cases, you understand performance isn’t the same [among providers] and you’re not looking for just price. You’re trying to find a balance,” he said.

Top-performing CSPs

When evaluating just performance, Cloud Spectator found that in certain scenarios, the Top 10-ranked providers exhibited a difference of more than 3.5x in CPU and memory performance, underscoring the need for performance testing to understand value.

Organizations often take performance for granted, Li said, and they shouldn’t.

“People think no matter what provider they choose, it doesn’t really matter. [They think they’re] kind of getting the same thing. But in reality, it’s not true at all,” he said. “When it comes to performance, it really varies from provider to provider.”

Key performance findings included the following:

• CenturyLink Inc. Cloud demonstrated the highest median VM performance within the top 10 best cloud providers.
• 1&1 Cloud demonstrated the best performance stability over the time period of the study.
• Rackspace Cloud demonstrated the highest median block storage performance within the top 10 best cloud providers.
• Nobis Technology Group, LLC’s Ubiquity Cloud and Google Compute Engine demonstrated the best block storage performance stability over the time period of the study.
• CSPs such as AWS and Google Compute Engine delivered IOPS proportional to the size of the block storage.

Pricing

Among Cloud Spectator’s top 10 CSPs, Interoute Communications Ltd., Rackspace and HostAway Pty Ltd had the highest monthly costs. Providers with the least expensive monthly costs were 1&1, Ubiquity Hosting and ProfitBricks.

Bandwidth pricing for egress traffic varies by CSP and by the amount of traffic. At the time of the study, 1&1 and Interoute, for example, charged nothing regardless of the amount, while Google charged $0.120 for the first GB, the first 5 GB and up to 1 TB and then lowers its pricing to $0.11 for 1 to 5 TB and 5 to 10 TB.

It may be tempting to select a provider based on price alone, but it is hazardous, Li warned.

“We’ve seen cases where people look at price alone and there’re under the assumption that when [they] move [their] four virtual core machines in [their] current environment over to another cloud provider, that it’s a one-to-one move. That’s not necessarily the case, though,” he said.

If performance is low, you will have to scale up your infrastructure to keep applications running at the level required.