This post continues our AI Factory series, which we started with the automated deployment of an AI Factory using OneDeploy. Its goal is to introduce the new deployment guide that shows how to run LLM models from Hugging Face using the vLLM appliance available in the OpenNebula Marketplace. The guide also walks through running a benchmark to validate the deployment and confirm that the environment is working as expected.

The benchmark shows how large language models perform on GPU-accelerated OpenNebula clouds, helping organizations understand what their infrastructure can realistically deliver before scaling real workloads. This makes it a practical and strategic enabler for enterprises building AI Factories, sovereign AI environments, or telco- and edge inference architectures.

LLM Benchmarking with OpenNebula

The Guide’s LLM-based inference benchmarks focus on measuring serving performance, rather than model quality, by evaluating key metrics such as latency, throughput, and stability under load. They are executed in a single-node GPU environment to eliminate orchestration noise and establish a clean, comparable performance baseline. Reference results are produced using certified hardware profiles, currently including NVIDIA L40S and H100 GPUs, and cover leading model families such as LLaMA and Qwen across multiple parameter sizes. A unified benchmarking workflow generates up to 1,000 parallel inference requests to measure latency, throughput, resource usage, and stability, resulting in a clear and reproducible performance report.

Beyond raw metrics, the benchmarks provide insight into time-to-first-token (TTFT), inter-token latency (ITL), responsiveness for real-time use cases, throughput for scaling, stability during long-running workloads, and GPU utilization efficiency for cost control. The results are complemented with SLO recommendations for common AI scenarios, such as chatbots, content generation, code assistants, and RAG- or agent-based pipelines, helping teams quickly determine whether their infrastructure is ready for production.

Why It Matters for Enterprise and Sovereign AI

The proposed validation process enables organizations to:

• Forecast capacity with confidence
• Optimize GPU spending
• Meet internal SLAs
• Reduce performance risks
• Deliver predictable AI service

Ready to validate your AI-ready Cloud?

Start with OpenNebula 7.0 and run your first benchmark today.

Stay tuned for more blogs and practical guides on building AI-ready infrastructure with OpenNebula