Booting the system with Upstart (II)

Continuing with the article Booting the system with Upstart (I) and in particular, the part of the events, we can also set many other types of triggers.

root@ubuntu-server:~# cat /etc/init/job1.conf
# Start when the loading modules have concluded
start on stopped load-modules

# Start when the filesystem is mounted
start on filesystem

# Start when the filesystem is mounted and the network is started
start on filesystem and started network

# Start under the runlevels 2, 3 and 5
start on runlevel [235]
...

The initctl utility allows to pass down a concrete event.

root@ubuntu-server:~# initctl emit myevent

If we take a look at the files dropped off within the /etc/init directory, we will be able to find out many other events. For instance, if we open the mountall.conf file, we can see that this job mounts the filesystems during the boot and emits events as local-filesystems, all-swaps, etc.

All the theory described for "start on", also can be applied when we want to stop a job.

root@ubuntu-server:~# cat /etc/init/job1.conf
stop on starting shutdown
...

This command as well can be used to list the status of all jobs handled by Upstart.

root@ubuntu-server:~# initctl list
mountall-net stop/waiting
rc stop/waiting
rsyslog start/running, process 687
...

Another interesting configuration option is the respawn tag, which is utilized to start automatically a job if it stops abnormally. We can also set that if a job is respawned more that X times in Y seconds, it will be stopped definitely.

root@ubuntu-server:~# cat /etc/init/job1.conf
respawn

respawn limit 10 120
...

What happens if we want to perform some previous task to start a job? For this case, Upstart provides the pre-start directive, which is run before the job reaches the starting state.

root@ubuntu-server:~# cat /etc/init/job1.conf
pre-start script
   rm /tmp/job*
end script
...

Instead of a script, we can also execute a binary.

root@ubuntu-server:~# cat /etc/init/job1.conf
pre-start exec /usr/local/sbin/nessusd
...

In addition, we can specify other kinds of pre/post actions.

root@ubuntu-server:~# cat /etc/init/job1.conf
# Before the job is started
post-start exec|script

# Before the job is stopping
pre-stop exec|script

# Before the job is stopped
post-stop exec|script
...

You can get more information as always... by accessing the man.

root@ubuntu-server:~# man 5 init

Taking snapshots on KVM with libvirt

There is an interesting feature related to the KVM virtualization, the snapshots, which take the disk, memory and device state of a concrete domain at a specific moment. This can have many use cases, such as saving a copy of a virtual machine, preserving a domain's state before a potentially dangerous operation, etc. Snapshots are identified with an unique name and can be taken with the virtual machine turned on.

This feature can only be used with the virsh command and not with the graphical tool (the version of Virtual Machine Manager included on Kubuntu 10.10 is the 0.8.4).

So as to create a snapshot for a particular domain, we must set a XML file with at least a name and description for the snapshot.

javi@javi-kubuntu:~$ cat UbuntuServer_10.10-ss.xml
<domainsnapshot>
   <name>UbuntuServer_10.10-16032011</name>
   <description>Snapshot of OS install and updates</description>
</domainsnapshot>

javi@javi-kubuntu:~$ virsh snapshot-create UbuntuServer_10.10 UbuntuServer_10.10-ss.xml
error: Requested operation is not valid: Disk '/var/lib/libvirt/images/UbuntuServer_10.10.img' does not support snapshotting

It is possible that as in the previous output, we get an error message about the virtual disk does not support snapshotting. What is the problem in this case?

The sort of the disk image is raw (default format when you create a virtual machine), which is very simple and does not allow snapshotting. In contradistinction to raw, qcow2 (QEMU image format) supports to have smaller images, encryption, compression, snapshots and so on.

In order to make a qcow2 virtual disk, we can run the following command. In this case, we have reserved all the space for the virtual disk. If we do not want to preallocate the image with metadata, we must take away the "preallocation" option.

javi@javi-kubuntu:~$ sudo qemu-img create -f qcow2 -o preallocation=metadata /var/lib/libvirt/images/UbuntuServer_10.10.qcow2 8G

javi@javi-kubuntu:~$ sudo chown libvirt-qemu:kvm /var/lib/libvirt/images/UbuntuServer_10.10.qcow2

If we want to convert an existing image (for example raw to qcow2), we can use the next order.

javi@javi-kubuntu:/var/lib/libvirt/images$ sudo qemu-img convert -f raw -O qcow2 -o preallocation=metadata UbuntuServer_10.10.raw UbuntuServer_10.10.qcow2

javi@javi-kubuntu:/var/lib/libvirt/images$ sudo chown libvirt-qemu:kvm UbuntuServer_10.10.qcow2

In this moment, we will be able to set a snapshot up with the virsh snapshot-create command. This operation will generate a new XML file.

javi@javi-kubuntu:~$ ls -l /var/lib/libvirt/qemu/snapshot/UbuntuServer_10.10/UbuntuServer_10.10-16032011.xml
-rw------- 1 root root 307 2011-03-16 11:40 /var/lib/libvirt/qemu/snapshot/UbuntuServer_10.10/UbuntuServer_10.10-16032011.xml

javi@javi-kubuntu:~$ virsh snapshot-dumpxml UbuntuServer_10.10 UbuntuServer_10.10-16032011
<domainsnapshot>
   <name>UbuntuServer_10.10-16032011</name>
   <description>Snapshot of OS install and updates</description>
   <state>shutoff</state>
   <creationTime>1300272022</creationTime>
   <domain>
      <uuid>d19af827-30ca-b3a0-8304-231f8cf5dd8b</uuid>
   </domain>
</domainsnapshot>

So as to list all the available snapshots for a certain domain, we can type the following order.

javi@javi-kubuntu:~$ virsh snapshot-list UbuntuServer_10.10
Name                        Creation time             Status
---------------------------------------------------------------
UbuntuServer_10.10-16032011 2011-03-16 11:40:22 +0100 shutoff

If at any point we want to bring back a snapshot, we must run the next command.

javi@javi-kubuntu:~$ virsh snapshot-revert UbuntuServer_10.10 UbuntuServer_10.10-16032011

And finally, we also have the option to remove a snapshot.

javi@javi-kubuntu:~$ virsh snapshot-delete UbuntuServer_10.10 UbuntuServer_10.10-16032011

System monitoring with nmon

Nmon is another interesting monitoring tool for Linux systems which can present many information related to the CPU, memory, network, etc. through an organized screen.

I have tested the 13g version on Ubuntu Server 10.10. When you start the application, this shows you a little menu with different options, in order to configure your own monitoring panel.

The next figure is a dump of my setting. I have used CPU utilization by processor, memory and swap stats, kernel stats and load average, network and disk input/output and top processes.




And finally, also say that with nmon you can take the data and dump them into a CSV file. For instance, in the following case I have run nmon in background to capture the data each 5 sg and a total of 200 times. Besides I have specified the file name with the '-F' option.

root@ubuntu-server:~# nmon -t -F `hostname`.csv -s 5 -c 200

Booting the system with Upstart (I)

Upstart is a management daemon which leads the starting and stopping of the system services, and besides, handles them whereas they are running. It is an application actually used on system such as Ubuntu, Fedora, RHEL, etc. and it has replaced the traditional System V init daemon (SysV).

It turns out that SysV has not been really taken away, since while there are tasks based on SysV, both systems will have to live together. Indeed, there is a job defined by Upstart and named rc-sysinit.conf, which controls the execution of the existing traditional SysV scripts. An Upstart system runs by default in runlevel 2.

root@ubuntu-server:~# runlevel
N 2

What is the main advantage of Upstart against SysV under my personal opinion? In contradistinction to SysV, Upstart allows to launch the services or tasks (here named jobs) in parallel, that is to say, they have just to wait for certain events happen. For example, when the Upstart init daemon has finished its initialization (upstart), when the filesystem is mounted (filesystem), MySQL daemon is started (started mysqld), and so on.

Let's see how Upstart works. All Upstart jobs are files made up of a well known structure and stored into the /etc/init directory. For example, we are going to develop an easy job to write the date within a tmp file. Its name will be job1.

root@ubuntu-server:~# cat /etc/init/job1.conf
script
   date > /tmp/job1.tmp ; sleep 10
end script

One of the ways to start this job is manually.

root@ubuntu-server:~# start job1
job1 start/running, process 1705

If we take a look at its state before 10 sg, we will see which is start/running. Afterwards, it will be stop/waiting, since in this case, it is not a daemon such as httpd, but a process which finishes when completes theirs defined tasks.

root@ubuntu-server:~# status job1
job1 start/running, process 1710

root@ubuntu-server:~# status job1
job1 stop/waiting

Upstart allows to define a job or by a script section (as the previous example - remember that the scripting code will be launch under "sh -e") either using the exec directive.

root@ubuntu-server:~# cat /etc/init/job1.conf
script
   exec date > /tmp/job1.tmp
end script

This sort of stanza can be used for instance to run a script or start a daemon like mysqld (zero or more arguments can be passed to it). Other manner to start a job is through an event. For example, we can run our job when job2 or job3 are started.

root@ubuntu-server:~# cat /etc/init/job1.conf
start on started job2 or started job3
...

By default, a job is a service that during its life cycle will be able to pass for different stages: starting, started, stopping and stopped, and in addition, will be able to be respawned if it exits with a zero status.

If we want to specify that the job is a task (it is considered to have finished when it has been run and stopped again - it will terminate with zero status but it will not be able to be respawned), we will have to add the task parameter into the configuration file. Otherwise, Upstart will treat the job as a service.