About 10 months ago our small ops team had to design a new infrastructure solution to run the services needed for our ecosystem. Until then we ran everything on root servers at a huge german webhoster. We knew that we wanted to go for our own infrastructure, but still be able to burst our load peaks into the cloud. We played around with several solutions on the market and finally wanted to meet some people at CEBIT 2013 to give us some hints for our decision.

To be honest we didn’t really consider OpenNebula at that time, but since the only person we could get hold of during lunchtime at CEBIT was Tino Vazquez from C12G we started to. A week later we fired up our first testing environment and 4 month later our productive OpenNebula Cluster started its work.

Why did we choose OpenNebula? Most of the software we use at runtastic is opensource and we wanted to set on KVM for that reason instead of some proprietary solution.  So we actually were looking for some management solution on top of KVM in the first step. However OpenNebula fullfills our requirements in several other terms as well.We wanted something simple and stable. We are a small team and have a lot of different services to maintain. We didn’t want to have an additional spot we need to focus on. What we can say up to now: We use OpenNebula, thats it. No big issues, no big efforts in performance tuning. The infrastructure runs and runs and runs … Another thing was, that have a lot of ideas in extending OpenNebula (integrate LXC, trigger chef bootstrap from within OpenNebula, make use of Netapp cloning features, …). We know ruby. OpenNebula is written in ruby. Perfect match.

At the moment we have a pretty straight forward solution based on two Netapp 2240 providing NFS shared storage and 28 Cisco UCS Servers running KVM on Ubuntu 12.04. Everything is connected via 10G Cisco Nexus Switches. We still have bare metal hardware for our databases (MySQL, MongoDB and Cassandra) and the services needed for OpenNebula itself, but everything else, like webservers, mobile endpoints and backend services is running in our cloud. All in all we run about 280 quite big (compared to a typical AWS machine) VMs consuming about 200 cores, 4TB memory and more than 20000 IOPS in average.

Everything in our infrastructure, also OpenNebula and KVM is set up automatically with Chef, so for us it is very easy to add new nodes to our cluster, or create new datastores.

Next step will be to integrate the VMware vCloud based IAAS solution of our datacenter provider to cover load peaks.