Proxmox Virtualization Environment

Proxmox VE is an open source virtualization platform based on Linux and used to run virtual appliances and virtual machines. It is a product aimed at companies and production environments, since it intends to be an easy deployment and management solution.

Proxmox VE supports three types of virtualization technologies:

• Container virtualization (OpenVZ): it allows that a physical server can run multiple insolated operating system instances as well known as containers. Its main problem is these containers or guests can just be Linux instances. However, the OpenVZ kernel provides better performance (penalty between 1% and 3% compared to a standalone server) in contradistinction to other alternatives.


• Full virtualization (KVM): it can run Linux and Windows guests but unlike OpenVZ, Intel VT or AMD-V CPU is needed.


• Paravirtualization (KVM): it presents a software interface similar to the underlying hardware in order to try to reduce the execution time of certain operations.

In my personal opinion, I think that OpenVZ is a plus because it has got better performance than KVM and the containers idea is fantastic: the size of these templates is really small and you can get a ready virtual machine in few minutes.

You can install Proxmox VE from an ISO image or directly on an existing Debian distribution (32 or 64 bits). I have tested Proxmox VE 1.7 (bare-metal ISO installer based on Debian Lenny) on a KVM virtual machine under my Kubuntu 10.10.

The process is very simple. The Proxmox installer is based on a graphical wizard with several stages: location and time zone selection, password and email address definition, and network configuration.

What are the main features of this product?

• Web based administration: easy deployment and management (web based management and virtual console, backup and restore with LVM2 snapshots, etc.).


• Virtual appliances: they are fully pre-installed and pre-configured applications including the operating system environment. You can create your own container, get from the community, use Linux OS instances or buy certified appliances.


• Proxmox VE cluster: it allows to gather multiple physical server in one VE cluster (central web management and login, cluster synchronization, easy cluster setup, live migration of virtual machines between physical servers, etc.).

When you finish the installation process, you must reboot the machine and update the system.

proxmox:~# aptitude update ; aptitude dist-upgrade

In order to manage Proxmox VE, you must open a web browser and type the IP address configured during the wizard (the default user is 'root').




The Proxmox web interface is very useful. It is formed by three principal sections, VM Manager, Configuration and Administration.

In the VM Manager area, you can upload ISO images and OpenVZ templates, download certified appliances and create and handle virtual machines.

In the Configuration area, you can set up the different parameters of the system (network, DNS, time, administrator options, language, proxy, etc.), add and manage data storages (ISCSI targets, NFS shares, LVM groups and directories) and create new backup jobs.

And finally, in the Administration area you can control the Proxmox VE certificates and services (ClusterSync, ClusterTunnel, NTP, SMTP, SSH and WWW), take a look at the logs and monitor the cluster nodes.

Frag needed and DF set

I was remembering a curious problem that we had at work the last year.

There was an HTTPS service that when you tried to access it by means of a web browser, the screen did not show anything. The display was blank waiting...

The main inconvenience was that we had not access to the web server to check it out. At that moment, I ran a tcpdump on the client in order to capture all network traffic and try to find out what was happening.

When I analyzed the packets I could see that the TCP connection was established correctly but then, the web server was sending frames with "TCP Previous segment lost", "TCP Dup ACK", "TCP Retransmission" messages. It is a pity because I do not keep these network captures to output them here...

Well, then it was clear that there was some device in the middle of the route between client and server, which was causing a leak of network packets in that communication.

Our network architecture was similar to the schema of the following image.




Taking a look at the firewall logs, we could see that there were ICMP packets related to the problematic HTTPS connection which were being dropped.

The router was sending ICMP packets (type 3, destination unreachable - code 4, fragmentation needed) to the source, in order to warn it that its MTU was smaller than the size of the packets, and on top of all that, the DF (prohibit fragmentation) was set to 1.

When that ICMP packet reached to the firewall, it was dropped and the source never knew that it had to reduce the data field for the TCP/IP packets. The solution was to allow the ICMP traffic (ICMP protocol with type 3 and code 4).

We can realize a little test in our computer. For example, attempting to send a packet whose size exceeds our network MTU (1500) and besides, DF=1 (192.168.1.100 is my PC and 192.168.1.1 the destination).

javi@kubuntu:~$ ping -c 1 -s 2000 -M do 192.168.1.1 PING 192.168.1.1 (192.168.1.1) 2000(2028) bytes of data.
From 192.168.1.100 icmp_seq=1 Frag needed and DF set (mtu = 1500)

If I run a tcpdump, I can see that I receive an ICMP datagram noting me that I have to lower the size of the packets.

javi@kubuntu:~$ sudo tcpdump -ni lo icmp -s0
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on lo, link-type EN10MB (Ethernet), capture size 65535 bytes
18:59:33.014773 IP 192.168.1.100 > 192.168.1.100: ICMP 192.168.1.1 unreachable
- need to frag (mtu 1500), length 556

Linux load average

I wanted to write an article about a topic that over time, it is still generating much confusion: the load average.

The system load average is a set of three numerical values which are provided by tools such as uptime or top. These values represent the average number of system processes that during the last 1, 5 and 15 minutes have been waiting for any resource of the system (CPU, disk access, network, etc.)

[root@centos ~]# uptime
15:43:45 up 9 days,  5:19,  1 user,  load average: 1.62, 1.49, 1.39

[root@centos ~]# top
top - 15:44:32 up 9 days,  5:20,  1 user,  load average: 1.38, 1.43, 1.37
...

In the previous output of the uptime command, 1.49 means that during the last 5 minutes, an average of 1.49 processes have been blocked waiting for some resource allocation.

For instance, I usually set a trigger in Zabbix for when the average load during the last 5 minutes has been higher than the number of cores available on the monitored machine.

When an alarm of this type is raised, does not necessarily have to mean that the CPU is overloaded. At that moment we have to use other Linux tools, such as top, vmstat, iostat, vnstat, etc, in order to find out what process or processes are involved and what resources are affected.

Finally, to emphasize the word of the title (Linux...), since these values for UNIX systems represent the number of processes which have utilized (running) the CPU or have been expecting for it (runnable).

KVM virtualization on Ubuntu (IV)

This is the last article about KVM virtualization on Ubuntu (I, II, III).

In the first post, I choose as hypervisor localhost (QEMU). If you pick out the other hypervisor available (localhost (QEMU Usermode)) and try to set up a virtual machine, in the last step you will get an error with the following text: "Imposible completar la instalación: «internal error Process exited while reading console log output: char device redirected to /dev/pts/1 qemu: could not open disk image /var/lib/libvirt/images/Ubuntu.img: No such file or directory".




The problem is when you select this hypervisor, the virtual machine is created with your user account, and if you take a look at the permissions on the /var/lib/lubvirt/images directory, you will see that only the root user can write in it.

javi@javi-kubuntu:~$ ls -l /var/lib/libvirt/
total 12
drwxr-xr-x 2 root         root 4096 2010-09-25 00:52 boot
drwxr-xr-x 2 root         root 4096 2010-09-25 00:52 images
drwxr-xr-x 4 libvirt-qemu kvm  4096 2010-12-07 12:28 qemu

The solution consists in changing the directory from root to libvirtd group and adding write permissions.

javi@javi-kubuntu:~$ sudo chown :libvirtd /var/lib/libvirt/images

javi@javi-kubuntu:~$ sudo chmod g+w /var/lib/libvirt/images

Remember that with this sort of connection (localhost (QEMU Usermode)), the storage will be into the /var/lib/libvirt/images directory, but the configuration and log files will reside in the user home.

javi@javi-kubuntu:~$ tree .libvirt/
.libvirt/
├── qemu
│   ├── cache
│   ├── lib
│   ├── log
│   │   └── Ubuntu.log
│   ├── run
│   ├── save
│   ├── snapshot
│   └── Ubuntu.xml
└── storage
├── autostart
│   └── default.xml -> /home/javi/.libvirt/storage/default.xml
└── default.xml

During four articles, we have learnt how to make virtual machines with KVM, libvirt and Virtual Machine Manager. I still have to present several articles related to snapshots, access to hypervisors running on remote machines, types of virtual networks, etc.

KVM is a great option to handle virtual machines. It would be a good idea to carry out some tests in order to meter the performance of different types of hypervisors, such as Xen or VMware.