Support: 1-800-961-4454
Sales Chat
1-800-961-2888

Architecting VMware vSphere For OpenStack

5

One of the most common questions I hear is “How does OpenStack compare with VMware?” That question comes not only from customers, but from my peers in the vendor community, many of whom have lived and breathed VMware technology for a number of years. Related questions include, “Which is a better cloud platform?” “Who has the better hypervisor?” and “Why would I choose one vs. another?” Many of these are important and sound questions; others, however, reveal a common misunderstanding of how OpenStack matches up with VMware.

The misunderstanding comes when people fail to differentiate between vSphere and other VMware technologies such as vCloud Director. It is not enough to just report that a company is running VMware; are they just running vSphere or are they also using other VMware products such as vCloud Director, vCenter Operations Manager or vCenter Automation Center? If a business announces it is building an OpenStack private cloud, instead of using the VMware vCloud Suite, does it necessarily imply that it is throwing out vSphere as well? The questions to be asked in such a scenario are “Which hypervisors will be used with OpenStack?” and “Will the OpenStack cloud be deployed alongside a legacy vSphere environment?”

What makes the first question particularly noteworthy is the fact that VMware has stepped up its commitment to OpenStack, with the Grizzly release, and the amount of code that it has recently contributed. Ironically, the OpenStack drivers for ESX were initially written not by VMware, but by Citrix. These drivers provided limited integration with OpenStack Nova Compute and were, understandably, not well maintained. However, with VMware now a member of the OpenStack community, its OpenStack team has been busy updating and contributing new code. Deeper integration between vSphere and OpenStack is important for users who want to move to an open cloud platform, such as OpenStack, but have existing investments in VMware and legacy applications that run on the vSphere platform.

In this post, I’ll provide an overview of the OpenStack Nova Compute architecture and how vSphere integrates with that project.

OpenStack Architecture

OpenStack is built on a shared-nothing, messaging-based architecture with modular components that each manage a different service; these services, work together to instantiate an IaaS Cloud.

 

openstack-arch-grizzly-v1-logical

A full discussion of the entire OpenStack architecture is beyond the scope of this post. For those who are unfamiliar with OpenStack or need a refresher, I recommend reading Ken Pepple’s excellent OpenStack overview blog post and my “Getting Started With OpenStack” slide deck. I will focus, in this post, on the compute component of OpenStack, called Nova.

Nova Compute

Nova compute is the OpenStack component that orchestrates the creation and deletion of compute/VM instances. To gain a better understanding of how Nova performs this orchestration, you can read the relevant section of the “OpenStack Compute Administration Guide.” Similar to other OpenStack components, Nova is based on a modular architectural design where services can be co-resident on a single host or, more commonly, on multiple hosts.

Nova

The core components of Nova include the following:

  • The nova-api accepts and responds to end-user compute API calls. It also initiates most of the orchestration activities (such as running an instance) as well as enforcing some policies.
  • The nova-compute process is primarily a worker daemon that creates and terminates virtual machine instances via hypervisor APIs (XenAPI for XenServer/XCP, libvirt for KVM or QEMU, VMwareAPI for vSphere, etc.).
  • The nova-scheduler process is conceptually the simplest piece of code in OpenStack Nova: it takes a virtual machine instance request from the queue and determines where it should run (specifically, which compute node it should run on).

Although many have mistakenly made direct comparisons between OpenStack Nova and vSphere, that is actually quite inaccurate since Nova actually sits at a layer above the hypervisor layer. OpenStack in general and Nova in particular, is most analogous to vCloud Director (vCD) and vCloud Automation Center (vCAC), and not ESXi or even vCenter. In fact, it is very important to remember that Nova itself does NOT come with a hypervisor, but manages multiple hypervisors, such as KVM or ESXi. Nova orchestrates these hypervisors via APIs and drivers. The list of supported hypervisors include KVM, vSphere, Xen and others; a detailed list of what is supported can be found on the OpenStack Hypervisor Support Matrix.

 

Screen Shot 2013-06-24 at 1.19.41 PM

Nova manages its supported hypervisors through APIs and native management tools. For example, Hyper-V is managed directly by Nova and KVM is managed via a virtualization management tool called LibVirt, while Xen and vSphere can be managed directly or through management tools such as LibVirt and vCenter for vSphere, respectively.

vSphere Integration with OpenStack Nova

OpenStack Nova compute manages vSphere 4.1 and higher through two compute driver options provided by VMware – vmwareapi.VMwareESXDriver and vmwareapi.VMwareVCDriver:

  • The vmwareapi.VMwareESXDriver driver, originally written by Citrix and subsequently updated by VMware, allows Nova to communicate directly to an ESXi host via the vSphere SDK.
  • The vmwareapi.VMwareVCDriver driver, created by VMware for the Grizzly release, allows Nova to communicate with a VMware vCenter server managing a cluster of ESXi hosts.

Let’s talk more about these drivers and how Nova leverages them to manage vSphere. Note that I am focusing specifically on compute and tabling discussions regarding vSphere networking and storage to other posts.

ESXi Integration with Nova (vmwareapi.VMwareESXDriver)

 

vmwareapi_blockdiagram

Logically, the nova-compute service communicates directly to the ESXi host; vCenter is not in the picture. Since vCenter is not involved, using the ESXDriver means advanced capabilities, such as vMotion, High Availability and Dynamic Resource Scheduler (DRS), are not available. Also, in terms of physical topology, you should note the following:

  • Unlike Linux kernel-based hypervisors, such as KVM, vSphere with OpenStack requires the VM instances to be hosted on an ESXi server distinct from a Nova compute node, which must run on some flavor of Linux. In contrast, VM instances running on KVM can be hosted directly on a Nova compute node.
  • Although a single OpenStack installation can support multiple hypervisors, each compute node will support only one hypervisor. So any multi-hypervisor OpenStack cloud requires at least one compute node for each hypervisor type.
  • Currently, the ESXDriver has a limit of one ESXi host per Nova compute service.

ESXDriver

 

ESXi Integration with Nova (vmwareapi.VMwareVCDriver)

Logically, the nova-compute service communicates to a vCenter Server, which handles management of an ESXi cluster. With vCenter in the picture, using the VCDriver means advanced capabilities, such as vMotion, High Availability and Dynamic Resource Scheduler (DRS), are now supported. However, since vCenter abstracts the ESXi hosts from the nova-compute service, the nova-scheduler views the cluster as a single host with resources amounting to the aggregate resources of all ESXi hosts managed by the cluster. This can cause issues with how VMs are scheduled/distributed across a multi-compute node OpenStack environment; please note, there are tentative plans to change this so that Nova can manage an ESXi cluster directly. I’ll devote an upcoming post to how VM scheduling works in OpenStack.

Also, in terms of physical topology, you should note the following:

  • Unlike Linux kernel-based hypervisors, such as KVM, vSphere with vCenter on OpenStack requires a separate vCenter Server host and that the VM instances to be hosted in an ESXi cluster run on ESXi hosts distinct from a Nova compute node. In contrast, VM instances running on KVM can be hosted directly on a Nova compute node.
  • Although a single OpenStack installation can support multiple hypervisors, each compute node will support only one hypervisor. So any multi-hypervisor OpenStack cloud requires at least one compute node for each hypervisor type.
  • Currently, the VCDriver has a limit of one vSphere cluster per Nova compute service. It is, therefore, recommended that the compute node run as a VM in that same cluster, with HA enabled.
  • Currently, the VCDriver requires that only one datastore can be configured and used per cluster.

VCDriver

The Rackspace View

Hopefully, this post has helped shed some light on where vSphere integration currently stands with OpenStack. The Rackspace Private Cloud, powered by OpenStack, is a supported deployment and configuration of OpenStack that leverages our expertise in operating the world’s largest open cloud. As such, we place a high value on the production readiness of any new technology before choosing to support it in our private cloud offering. So while we support all of the OpenStack projects, we don’t necessarily support every feature at time of release. In the case of Nova Compute, Rackspace has chosen to support KVM as the hypervisor in our private cloud while continuing to evaluate other hypervisors. The original VMwareESXDriver implementation of ESXi, while useful for proving out the multi-hypervisor capabilities of OpenStack, is not suitable for production usage. However, the VMwareVCDriver implementation has great potential and is something that is being evaluated seriously.

Want to know more about Kenneth Hui and his thoughts on OpenStack? Check out this Q&A interview.

Heading to VMworld 2013, August 25 to August 29 in San Francisco? We are too.

Come check us out at booth No. 2404 on the expo floor – we’re excited to talk to you about new additions to our VMware® product line and the future of the Rackspace Hybrid Cloud. We’ll also have technical experts in the booth to answer all of your questions about everything from Rackspace Replication Manager (powered by VMware SRM to VMware to OpenStack.

While you’re here, play our “Unlock the Cloud” game for a chance to win prizes.

And be sure to see Rackspace Enterprise Cloud Strategist Paul Croteau and VMware’s Bryan Evans present “DR to the Cloud with VMware Site Recovery Manager and Rackspace Disaster Recovery Planning Services” at 4 p.m. PDT on Monday, August 26.

About the Author

This is a post written and contributed by Rackspace .


More
5 Comments

Your link to a slide deck above ends up at a page saying the deck has been removed…

avatar Brian Gilstrap on August 27, 2013 | Reply

Hi Brian,

I’m looking into it now.

Thanks,
-Andrew

avatar Andrew Hickey [Racker] on August 27, 2013

Hi Brian,

That deck had been updated recently. Here’s the new link (it’s also been updated in the body of the blog post):

http://www.slideshare.net/kenhui65/getting-started-with-openstack-20130821-25506318

Thanks,
-Andrew

avatar Andrew Hickey [Racker] on August 27, 2013

Kenneth, incredibly useful post. Im looking into OStack for my organization, and since we have Cisco appliances that only run on ESXi as the host, I was in doubt about the extent of functionality we could obtain from a ESXi/OStack combination.

Please keep us updated on this line: “there are tentative plans to change this so that Nova can manage an ESXi cluster directly”

Thank you!

avatar Jose Mendez on August 28, 2013 | Reply

Great stuff Ken. Very thorough and informative. Helped me answer some hot topic OpenStack / VMware integration questions.

Thank you very much.

jtg

avatar Jason Grimm on October 22, 2013 | Reply

Leave a New Comment

(Required)


Racker Powered
©2014 Rackspace, US Inc.