In Cloud Management Platforms users typically deploy their VMs out of pre-defined templates that can specify a fixed amount of memory. Users may be able to customize the size of their VMs but they tend to overestimate the memory required for their applications. As an example, three 4 GB VMs with applications actually using 1 GB of RAM fit in a 12 GB node, but there is no more room left for additional VMs. Adjusting their memory to 1 GB VMs enables to deploy additional VMs on that node, as shown in the next figure.

CloudVAMP is an open-source development that manages these situations for all the nodes in an OpenNebula Cloud deployment based on the KVM hypervisor. CloudVAMP monitors the memory usage of Virtual Machines (VMs) and dynamically changes the memory allocated to VMs by stealing the unused free memory from VMs. Then CloudVAMP enables OpenNebula to use that stolen memory, thus being able to increase the VM-per-node ratio. To prevent memory overload in the physical hosts, live migration is applied in order to accommodate the increasing memory demand by VMs across the OpenNebula Cloud.

Some Technical Details

CloudVAMP consists of three components:

• Cloud Vertical Elasticity Manager (CVEM). An agent that analyzes the amount of memory actually needed by the VMs and dynamically updates the memory allocated to each of them, according to a set of customizable rules.
• The Memory Reporter (MR). An agent that runs in the VMs and reports to the OpenNebula monitoring system the free, used memory and usage of the swap space, by the applications in the VM.
• The Memory Oversubscription Granter (MOG). A system that informs OpenNebula about the amount of memory that can be oversubscribed from the hosts, to be taken into account by the OpenNebula scheduler.

CloudVAMP integrates with OpenNebula in several ways. The MR can be staged in the VMs using the contextualization mechanisms provided by OpenNebula or it can be pre-installed in the Virtual Machine Images. It contacts OneGate to report the memory usage in the VMs. The CVEM is installed as a daemon in the front-end node of the OpenNebula Cloud. Finally, the MOG is implemented as a new Information Manager in OpenNebula. The interaction with KVM is performed by means of LibVirt. Therefore, no modifications in the OpenNebula worker nodes are required. The interaction with the components is shown in the following figure:

Benefits of Using CloudVAMP

Deploying CloudVAMP in an OpenNebula Cloud enables to seamlessly allow OpenNebula to deploy more VMs per physical host, thus achieving increased server density. The memory usage of VMs is monitored in order to satisfy increased memory demands by the applications running in the VMs. The usage of live migration to redistribute the VMs without downtime is employed if necessary, without any user or sysadmin intervention. This enables an increased usage of the hardware platform that supports an OpenNebula Cloud.

In particular, at the GRyCAP research group we have integrated CloudVAMP in order to accommodate a larger number of incoming jobs from the ES-NGI (the Spanish National Grid Initiative) that are executed on a virtual elastic cluster deployed and managed by EC3 (Elastic Cloud Computing Cluster). The virtual cluster, deployed on top of our OpenNebula Cloud, is horizontally scaled whenever incoming jobs are received (i.e., deploying additional Worker Nodes (WNs)) and vertically scaled (i.e., adjusting the allocated memory to the VMs) in order to let OpenNebula deploy additional WNs in the same host, if necessary. Further details of this case study are available in CloudVAMP’s reference publication.

Availability

CloudVAMP has been developed by the GRyCAP research group at the Universitat Politècnica de València. It is available under the Apache 2.0 license at GitHub.

There is further information in CloudVAMP’s web page and in the corresponding publication:

Germán Moltó, Miguel Caballer, and Carlos de Alfonso. 2016. “Automatic Memory-Based Vertical Elasticity and Oversubscription on Cloud Platforms.” Future Generation Computer Systems 56: 1–10. http://linkinghub.elsevier.com/retrieve/pii/S0167739X15003155.

Contributions, feedback and issues are very much welcome.