IBM Research Division has just published a paper detailing a side-by-side comparison of Docker containers versus the KVM hypervisor, highlighting the latter’s cost when used with NAT or AUFS, and questioning the wisdom of running containers inside virtual machines.

Big Blue’s boffins compared the two technologies using the linear-equation solving package Linpack, network bandwidth using nuttcp, the STREAM benchmark of memory bandwidth and latency using netprf. Benchmarks also include sample Redis and MySQL workloads; Redis exercises the networking stack, with small packets and a large number of clients, while MySQL stresses memory, network and the filesystem.

The resulting paper, An Updated Performance Comparison of Virtual Machines and Linux Containers, reveals that Docker “equals or exceeds KVM performance in every case tested”. With regard to CPU and memory performance, both Docker and KVM introduced a measurable but negligible overhead, although for I/O intensive applications both technologies required tuning.

You could be forgiven for thinking that it’s time to kiss goodbye to virtualization already, given that Docker version 1.0 is already beating its long-established rival. But IBM’s boffins urge that caution is the order of the day. They note there’s been “continuous improvement” in KVM’s performance over the years, while for containers “things can only get worse because they started with near-zero overhead and VMs have gotten faster over time.”

Docker isn’t entirely perfect anyway, as the researchers note its performance degrades when using files stored in AUFS, compared to using volumes, that have better performance. They also said its network address translation makes for extra traffic within networks.

However, the boffins do question the implementation of IaaS using virtual machines:

“Conventional wisdom (to the extent such a thing exists in the young cloud ecosystem) says that IaaS is implemented using VMs and PaaS is implemented using containers. We see no technical reason why this must be the case, especially in cases where container based IaaS can offer better performance or easier deployment. Containers can also eliminate the distinction between IaaS and “bare metal” non-virtualized servers since they offer the control and isolation of VMs with the performance of bare metal.”

So IBM says it’s actually better to run containers in bare metal Linux servers, contrary to the common practice of running them inside virtualized environments. If run inside a virtual machine, containers are bound by the VM’s performance overheads with no added benefit to using containers in a non-virtualized Linux.

If anyone would like to run the tests and see for themselves, the scripts for IBM’s study can be found here.

photo credit: Anant N S (www.thelensor.tumblr.com) via photopin cc