While most enterprises seem to be racing headlong into the cloud as fast as they can, a few have come to the realization that cloud computing isn’t always the Holy Grail for all their infrastructure needs.

GitLab Inc., the software developer platform that rivals the better-known GitHub Inc., has become the latest to revert back to on-premises infrastructure, following Dropbox Inc. earlier this year. Dropbox raised quite a few eyebrows when it said it had “mostly quit” Amazon Web Services’ cloud in favor of its own data centers.

Just like Dropbox, GitLab reckons that public clouds just don’t have what it takes to scale beyond a certain point. In a blog post, GitLab’s infrastructure chief Pablo Carranza revealed the cloud just wasn’t reliable enough when running the Ceph Filesystem, which uses a cluster running the Ceph objects-and-blocks-and-files storage platform.

He explained that Ceph FS requires a “really performant underlaying infrastructure because it needs to read and write a lot of things really fast,” and that the cloud just wasn’t up to the job.

“If one of the hosts delays writing to the journal, then the rest of the fleet is waiting for that operation alone, and the whole file system is blocked. When this happens, all of the hosts halt, and you have a locked file system; no one can read or write anything and that basically takes everything down,” he explained.

Carranza explained further, saying that when you get into the consistency, accessibility and partition tolerance of CephFS, public clouds just give away availability in exchange for consistency. Another problem he identified is that when the system is put under too much pressure, it generates “hot spots” wherein all the reads and writes are performed on the same spot during high load times, thereby slowing everything down.

In essence, GitLab’s use of Ceph FS meant it had a “noisy neighbor” problem that hogs all of the cloud server’s resources, severely affecting performance elsewhere. That led Carranza to conclude that the cloud just isn’t capable of providing the level of IOPS performance, a measurement of the speed of data exchange in computer storage devices, that an aggressive system like CephFS needs.

But Carranza wasn’t done yet, because he had some harsh words for the cloud in general:

“At a small scale, the cloud is cheaper and sufficient for many projects. However, if you need to scale, it’s not so easy. It’s often sold as, “If you need to scale and add more machines, you can spawn them because the cloud is ‘infinite.'” What we discovered is that yes, you can keep spawning more machines but there is a threshold in time, particularly when you’re adding heavy IOPS, where it becomes less effective and very expensive. You’ll still have to pay for bigger machines. The nature of the cloud is time sharing so you still will not get the best performance. When it comes down to it, you’re paying a lot of money to get a subpar level of service while still needing more performance.

As such, GitLab has decided to dump its unnamed cloud provider in exchange for dedicated on-premises hardware, which Carranza says is “more economical and reliable because of how the culture of the cloud works and the level of performance we need.”

Photo Credit: leo2109 Flickr via Compfight cc