One of our main use cases is enabling the growing number of companies looking to build private and hybrid cloud infrastructures for running AI training and inference services. Compared to relying on public cloud providers, deploying a private cloud for AI brings several key benefits:

• Cost Efficiency: Owning and managing infrastructure significantly reduces long-term costs compared to public cloud services.
• Data Privacy and Security: Full control over data ensures regulatory compliance and eliminates concerns about third-party access.
• Customizability: Tailor the environment to optimize AI frameworks, libraries, and tools for maximum efficiency.
• Vendor Neutrality: Avoid vendor lock-in by leveraging open source solutions and diverse hardware options.
• Performance Optimization: Fine-tune infrastructure to meet workload-specific requirements.
• Reduced Latency: Locally hosted infrastructure minimizes latency for real-time AI inference and data processing.

OpenNebula: Powering Multi-Tenant AI Factories

To support this vision, OpenNebula offers key features for multi-tenant AI cloud environments, including:

• Robust Multi-Tenancy: Secure resource sharing across teams while efficiently leveraging GPU acceleration.
• High-Performance Hardware Access: Support for SR-IOV and PCI passthrough enables direct hardware access, optimizing GPU-intensive workloads.
• True As-a-Service Model: Seamlessly manage on-premise and multi-cloud hybrid AI deployments.

New AI Integration: Deploying LLMs on OpenNebula

Over the past months, we have been working to simplify the deployment and orchestration of Large Language Models (LLMs) from Hugging Face within OpenNebula-powered infrastructure.

As part of OneApps 6.10.0-3, we have released the Ray Appliance, designed for managed AI inference and LLM applications. This integration with Hugging Face allows users to:

• Easily deploy AI applications within their cloud infrastructure.
• Leverage OpenNebula’s scalability and automation for AI workloads.

The app has been tested using several LLMs, like Llama from Meta, Qwen from Alibaba, Mistralai, EuroLLM, ALIA and many others. 

Check out this new screencast showcasing how to automatically deploy AI LLMs from Hugging Face on an OpenNebula-powered cloud using the Ray Appliance from the OpenNebula App Marketplace.

This is the initial version, and we are actively enhancing the appliance to support not just LLMs but also various types of ML models, including classification, sentiment analysis, time-series, and more. Additionally, we are working on providing clear guidelines and examples for training or fine-tuning other models, as well as instructions on how to deploy and use your own models through this appliance.

Join Our Webinar: Empowering AI in the Cloud

We’re hosting a webinar where we will:

• Present our first Hugging Face integration
• Demonstrate the Ray Appliance in action
• Answer live questions from the community

Meet Us at MWC25: Build Your AI-Ready Telco Infrastructure

We will also be running an “AI-Ready Booth: Build Your AI-Ready Telco Infrastructure for the Future” at Mobile World Congress 2025 in Barcelona. Secure your one-on-one meeting!

What’s Next?

This is just the beginning! We are continuously enhancing our AI capabilities, adding new features, integrations, and partnerships to bring AI to your data center. In particular, the next version of the AI appliance will include:

• Support to train, fine-tune, and deploy models seamlessly as part of AI workflows.
• Support for deploying vLLMs
• Support for exposing the inference endpoints with OpenAI API
• Support for using multiple GPUs/vGPUs

Stay tuned—exciting times ahead!

---

---