Adding a VMware ESXi hypervisor to OpenNebula (II)

Let's go ahead with the development of the cloud computing infraestructure based on OpenNebula by adding a VMware ESXi hypervisor. In the previous article, we saw how to configure a VMware vSphere node and now, we are going to set up the needed part on frontend01.

In order to manage the VMware node from OpenNebula, it is necessary to install libvirt on frontend01, and on top of all that, this software must be compiled with the ESX support. In this way, you cannot use the corresponding package located in the Ubuntu repositories. I am going to utilize the 0.9.2 version of libvirt.

root@frontend01:~# aptitude install libgnutls-dev libdevmapper-dev libcurl4-gnutls-dev python-dev libnl-dev libapparmor-dev

root@frontend01:/tmp# wget http://libvirt.org/sources/libvirt-0.9.2.tar.gz

root@frontend01:/tmp# tar xvzf libvirt-0.9.2.tar.gz ; cd libvirt-0.9.2

root@frontend01:/tmp/libvirt-0.9.2# ./configure --with-esx --with-apparmor --sysconfdir=/etc --libdir=/usr/lib --sbindir=/usr/sbin --datarootdir=/usr/share --localstatedir=/var --libexecdir=/usr/lib/libvirt

root@frontend01:/tmp/libvirt-0.9.2# make ; make install

I have also compiled the source code with AppArmor support, so I have had to move the required files.

root@frontend01:/tmp/libvirt-0.9.2# mkdir -p /etc/apparmor.d/libvirt

root@frontend01:/tmp/libvirt-0.9.2# cp -a examples/apparmor/usr.* /etc/apparmor.d/
root@frontend01:/tmp/libvirt-0.9.2# cp -a examples/apparmor/TEMPLATE /etc/apparmor.d/libvirt/
root@frontend01:/tmp/libvirt-0.9.2# cp -a examples/apparmor/libvirt-qemu /etc/apparmor.d/abstractions/

root@frontend01:/tmp/libvirt-0.9.2# cat /etc/apparmor.d/usr.sbin.libvirtd
...
owner /srv/cloud/one/var/** rw,
}

root@frontend01:/tmp/libvirt-0.9.2# /etc/init.d/apparmor restart

Next step is to download and install the VMware Drivers Addon (version 2.2.0 in my case). This wrapper enables the communication between OpenNebula and VMware ESXi through libvirt. This operation must be effected by means of the oneadmin user.

When you run the install.sh script and if everything went well, you will get a message about a specific code that you have to add into the oned.conf file.

oneadmin@frontend01:/tmp$ wget http://dev.opennebula.org/attachments/download/350/vmware-2.2.0.tar.gz

oneadmin@frontend01:/tmp$ tar xvzf vmware-2.2.0.tar.gz ; cd vmware-2.2.0

oneadmin@frontend01:/tmp/vmware-2.2.0$ ./install.sh
VMWare Drivers Addon successfully installed

# After the installation, please add the following to your oned.conf file
# and restart OpenNebula to activate the VMware Drivers Addon

#-------------------------------------------------------------------------------
#  VMware Driver Addon Virtualization Driver Manager Configuration
#-------------------------------------------------------------------------------
VM_MAD = [
name       = "vmm_vmware",
executable = "one_vmm_sh",
arguments  = "vmware",
default    = "vmm_sh/vmm_sh_vmware.conf",
type       = "vmware" ]
#-------------------------------------------------------------------------------

#-------------------------------------------------------------------------------
#  VMware Driver Addon Information Driver Manager Configuration
#-------------------------------------------------------------------------------
IM_MAD = [
name       = "im_vmware",
executable = "one_im_sh",
arguments  = "vmware" ]
#-------------------------------------------------------------------------------

#-------------------------------------------------------------------------------
# VMware Driver Addon Transfer Manager Driver Configuration
#-------------------------------------------------------------------------------
TM_MAD = [
name       = "tm_vmware",
executable = "one_tm",
arguments  = "tm_vmware/tm_vmware.conf" ]
#-------------------------------------------------------------------------------

Due to there is a mistake in the install.sh script, you must execute manually the following two orders.

oneadmin@frontend01:/tmp/vmware-2.2.0$ mkdir -p $ONE_LOCATION/var/remotes/im/vmware.d && cp -r im/remotes/* $ONE_LOCATION/var/remotes/im/vmware.d

oneadmin@frontend01:/tmp/vmware-2.2.0$ mkdir -p $ONE_LOCATION/var/remotes/vmm/vmware && cp -r vmm/remotes/* $ONE_LOCATION/var/remotes/vmm/vmware

This problem causes that you cannot start the VMware node up and the log file will show the next error.

oneadmin@frontend01:~$ cat var/oned.log
...
[ONE][E]: syntax error, unexpected $end, expecting VARIABLE at line 2, columns 1:2

Before restarting OpenNebula, you must type the user and password used to access to esxi01 and include a line into the sudoers file, so that OpenNebula may properly set some permissions.

oneadmin@frontend01:~$ cat etc/vmwarerc
...
USERNAME      = "oneadmin"
PASSWORD      = "xxxxxx"

root@frontend01:~# cat /etc/sudoers
...
oneadmin ALL=NOPASSWD:/srv/cloud/one/share/hooks/fix_owner_perms.sh ""

root@frontend01:~# /etc/init.d/opennebula restart

So as to check the correct installation, run the following command. Thereby, you will be able to obtain the physical resources of the managed node.

oneadmin@frontend01:~$ lib/remotes/im/run_probes vmware esxi01
/srv/cloud/one/lib/ruby/vmwarelib.rb:26: warning: already initialized constant ONE_LOCATION
/srv/cloud/one/lib/ruby/vmwarelib.rb:32: warning: already initialized constant RUBY_LIB_LOCATION
HYPERVISOR=vmware TOTALCPU=100 CPUSPEED=3001 TOTALMEMORY=2096460 FREEMEMORY=1677168

Now if you want to add the VMware vSphere node configured to OpenNebula, you can execute the next order.

oneadmin@frontend01:~$ onehost create esxi01 im_vmware vmm_vmware tm_vmware

oneadmin@frontend01:~$ onehost list
ID NAME              CLUSTER  RVM   TCPU   FCPU   ACPU    TMEM    FMEM STAT
0  esxi01            default    0    100    100    100      2G    1.6G   on