Invalid or incomplete multibyte or wide character

The other day I downloaded a video with JDownloader and when I tried to move it into my usb memory, the system did not recognize the file and it could not move it.

The error message which Linux returned was "Caracter multibyte amplio inválido ó incompleto" (I have configured my system in Spanish), "Invalid or incomplete multibyte or wide character" in English.

javi@javi-kubuntu:~$ mv .jdownloader/downloads/TICs_espa�olas.avi /media/disk-2/
mv: no se puede efectuar «stat» sobre «/media/disk-2/TICs_espa\244olas.avi»: Caracter multibyte amplio inválido ó incompleto

This is a serious problem, because in addition to not be able to copy or move the file, you can neither remove it.

The issue is that file is encoded with ISO-8859-15, and my system is configured as UTF-8. The solution consists in converting the file name encoding to UTF-8. In order to carry out this operation, we can use the convmv utility.

javi@javi-kubuntu:~$ sudo aptitude install convmv

javi@javi-kubuntu:~/.jdownloader/downloads$ convmv -f iso-8859-15 -t utf-8 TICs_espa�olas.avi
Your Perl version has fleas #37757 #49830                                                                                                                                                                                                                                
Starting a dry run without changes...                                                                                                                                                                                                                                    
mv "./TICs_espa�olas.avi"    "./TICs_espa€olas.avi"                                                                
No changes to your files done. Use --notest to finally rename the files.

The previous output shows us the final file name after the conversion. So as to apply the changes permanently, you have to add the --notest argument.

javi@javi-kubuntu:~/.jdownloader/downloads$ convmv --notest -f iso-8859-15 -t utf-8 TICs_espa�olas.avi
Your Perl version has fleas #37757 #49830                                                                                                                                                                                                                                  
mv "./TICs_�olas.avi"    "./TICs_€olas.avi"                                                                
Ready!

KVM virtualization on Ubuntu (III)

Let's continue exploring more stuff about KVM virtualization on Ubuntu (remember that there have been posted two previous articles, I and II). We will start with an interesting characteristic: cloning existing virtual machine images with identical virtual hardware configurations.

This is a typical operation because it is normal that we have to install several times the same operating system for different functions. For example, we can need three servers running under a Linux distribution: mail, web and data storage. For this purpose, first we would make a virtual machine with that Linux OS, then we would clone that virtual machine twice and finally, we would install the rest of services on the base systems.

The following order generates a new virtual machine named cloned_BT, from BackTrack4 domain. It is necessary to specify the hypervisor by means of the --connect parameter and to have turned the virtual machine off.

javi@javi-kubuntu:~$ virt-clone --connect qemu:///system --original BackTrack4 --auto-clone --name cloned_BT

We can also perform this task through the Virtual Machine Manager.




Other interesting feature is related to the possibility of saving a running domain into a state file, with the aim of being able to be restored later. When we execute this operation, the domain will be automatically switched off.

javi@javi-kubuntu:~$ virsh save BackTrack4 BT4.save

javi@javi-kubuntu:~$ virsh restore BT4.save

We can use for example this characteristic to back up our virtual machine before a critical task, such as a system update. Therefore if we want to restore a domain from a state file, we will add the restore option (the virtual machine must be turned off).

And finally, I will present another useful tool in order to show the state of the virtualized domains. This utility is similar to top.

javi@javi-kubuntu:~$ sudo aptitude install virt-top

javi@javi-kubuntu:~$ virt-top --connect qemu:///system
virt-top 18:01:14 - x86_64 2/2CPU 2000MHz 3961MB
2 domains, 2 active, 2 running, 0 sleeping, 0 paused, 0 inactive D:0 O:0 X:0
CPU: 1,0%  Mem: 1024 MB (1024 MB by guests)

 ID S RDRQ WRRQ RXBY TXBY %CPU %MEM    TIME   NAME                                                                                                                                                                                                                         
  1 R    0    0   52    0  0,5 12,0   0:41.37 BackTrack4
  2 R    0    0   52    0  0,5 12,0   0:30.69 UbuntuServer_10.10

We can also get certain information about a concrete virtual machine selecting the Performance console, in the Virtual Machine Manager.


 

KVM virtualization on Ubuntu (II)

In the previous article about KVM virtualization on Ubuntu (I), I exposed how to set up virtual machines with KVM. Now we will learn how to handle them, either through the command line or by means of the Virtual Machine Manager graphical tool.

First of all, we must know where are located the most important files used by KVM (configuration, virtual images and log files). We can see that the characteristics of the virtual machines are stored into XML files.

javi@javi-kubuntu:~$ ls -lh /etc/libvirt/qemu/
total 12K
-rw------- 1 root root 2,0K 2010-10-12 23:00 BackTrack4.xml
drwxr-xr-x 3 root root 4,0K 2010-10-12 18:22 networks
-rw------- 1 root root 2,0K 2010-11-06 16:08 UbuntuServer_10.10.xml

javi@javi-kubuntu:~$ ls -lh /var/lib/libvirt/images/
total 21G
-rw------- 1 root root  12G 2010-10-12 23:02 BackTrack4.img
-rw------- 1 root root 8,0G 2010-11-06 17:04 UbuntuServer_10.10.img

javi@javi-kubuntu:~$ ls -lh /var/log/libvirt/qemu/
total 12K
-rw------- 1 root root    0 2010-10-13 21:38 BackTrack4.log
-rw------- 1 root root 2,3K 2010-11-06 17:04 UbuntuServer_10.10.log

It is also important to know the basic information about the hardware (number and type of CPU, size of the physical memory, etc.) where we will create the virtual machines.

javi@javi-kubuntu:~$ virsh nodeinfo
Modelo del CPU:      x86_64
CPU(s):              2
Frecuencia de CPU:   2000 MHz
Zócalo(s) de CPU:   1
Núcleo(s) por ranura: 2
Hilo(s) por núcleo: 1
Celda(s) NUMA:       1
Tamaño de memoria:  4056208 kB

Remember that in KVM, virtual machines are also known as domains. In order to list the state (running, idle, paused, shutdown, shut off, crashed or dying) of all existing domains, we must run the following order:

javi@javi-kubuntu:~$ virsh list --all
Id Nombre               Estado
----------------------------------
- BackTrack4           apagado
- UbuntuServer_10.10   apagado

If we want to edit the features (processor, memory, boot options, disk, NICs, monitor, etc.) of a concrete domain or add new virtual hardware components (storage, network, graphics, serial, parallel, watchdog, etc.), we can use Virtual Machine Manager.




Other way to modify the details of a virtual machine can be by means of the virsh command. This option is equivalent to output the domain information as an XML dump, edit that dump file, validate the XML file and define the domain from the XML file.

javi@javi-kubuntu:~$ virsh edit BackTrack4

javi@javi-kubuntu:~$ virsh dumpxml BackTrack4 > domain.xml
javi@javi-kubuntu:~$ vi domain.xml
javi@javi-kubuntu:~$ virt-xml-validate domain.xml
javi@javi-kubuntu:~$ virsh define domain.xml

Then we are going to present the necessary commands to start an inactive domain, suspend a running virtual machine in memory, move it out of the suspended state (resume), reboot it and shut it down.

javi@javi-kubuntu:~$ virsh start UbuntuServer_10.10

javi@javi-kubuntu:~$ virsh suspend UbuntuServer_10.10

javi@javi-kubuntu:~$ virsh resume UbuntuServer_10.10

javi@javi-kubuntu:~$ virsh reboot UbuntuServer_10.10

javi@javi-kubuntu:~$ virsh shutdown UbuntuServer_10.10

Other interesting option for a specific domain is dominfo, which returns basic information about the virtual machine.

javi@javi-kubuntu:~$ virsh dominfo BackTrack4
Id:             -
Nombre:         BackTrack4
UUID:           cc951529-f630-b8ea-1da8-9a75e382190d
Tipo de sistema operatuvo: hvm
Estado:         apagado
CPU(s):         1
Memoria máxima: 524288 kB
Memoria utilizada: 524288 kB
Persistente:    sí
Autoinicio:     desactivar
Modelo de seguridad: apparmor
DOI de seguridad: 0

And finally, if we want the domain to be automatically started at boot, we must run the next order. If we want to disable this option, we must add the --disable parameter.

javi@javi-kubuntu:~$ virsh autostart BackTrack4

javi@javi-kubuntu:~$ virsh autostart --disable BackTrack4

KVM virtualization on Ubuntu (I)

I do not like too much Oracle, I have to recognize it, particulary its business model. For that reason, I think that I have to try (whenever I can) to use other alternative products. The future of certain applications as OpenOffice, MySQL, VirtualBox is not clear...

In this article we are going to relate how to set up virtual machines on Ubuntu with KVM. I think that KVM is the future of the virtualization in the open source world, especially because it is a technology developed by Red Hat and this company is betting very hard for it.

First of all, we have to make sure whether our hardware supports virtualization (I am going to carry out the testing on Kubuntu 10.10 64 bits). For this purpose, we must check out if the vmx or svm flags appear into the cupinfo file.

javi@javi-kubuntu:~$ cat /proc/cpuinfo | egrep "(vmx | svm)" | wc -l
2

In my case, the result is two (my processor has got two cores). Then, we have to install the necessary packages:

javi@javi-kubuntu:~$ sudo aptitude install kvm libvirt-bin virt-manager

Now, we can see a new virtual network interface (virbr0) and a new group (libvirtd) in the system. My user name has also been added to the libvirtd group.

javi@javi-kubuntu:~$ ip a
...
19: virbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN
link/ether 0e:af:bc:3a:c8:6a brd ff:ff:ff:ff:ff:ff
inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
inet6 fe80::caf:bcff:fe3a:c86a/64 scope link
valid_lft forever preferred_lft forever

javi@javi-kubuntu:~$ cat /etc/group
...
libvirtd:x:124:javi

We have to restart the sesson for the user changes take effect. In order to verify that everything is right, we can type the following command:

javi@javi-kubuntu:~$ virsh -c qemu:///system list
Id Nombre               Estado
-------------------------------

Now we can create our first virtual machine opening the Virtual Machine Manager (Applications, System, Virtual Machine Manager), picking out localhost (QEMU) - Not Connected option and making double click on it (this is the hypervisor to which we connect - qemu:///system). Then we must press on Create a new virtual machine button and complete the different steps of the wizard.

Through the wizard, we will have to choose the features or resources for the virtual machine, such as the name, installation source (ISO, CD-ROM, network, PXE, importing the image, etc.), type and version of the operating system, amount of memory and CPUs, size of the virtual hard disk, etc.

In the last step, we can see a little summary about the selected characteristics and besides, pick out the type of virtualization (kvm, qemu - we will choose kvm), the sort of architecture (x86_64 or i686 - depends on the operating system to be installed) and the kind of network (NAT by default). In future articles, I will explain how to configure different types of virtual networks.




During the virtual store step, we can select a tab about allocating all the disk size now... In my personal opinion, we must mark this option because the virtual machine performance will be better if all the disk space is reserved from the beginning.

At the end of this process, we will have a screen with the new virtual machine embedded within it and we will be able to performe several operations on it: run, pause, shut down, clone, migrate, etc.