GPT, beyond the MBR (II)

Let's follow up on the second part of the article about GPT, beyond the MBR.

Now, a new GPT partition table is going to be created by means of parted, and afterwards, it will be displayed.

root@ubuntu-server:~# parted /dev/sdb mklabel gpt

root@ubuntu-server:~# parted /dev/sdb print
Model: VMware, VMware Virtual S (scsi)
Disk /dev/sdb: 1074MB
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number  Start  End  Size  File system  Name  Flags

If you try to edit the partition table with fdisk, you will come across a message as follows.

root@ubuntu-server:~# fdisk /dev/sdb 

WARNING: GPT (GUID Partition Table) detected on '/dev/sdb'! The util fdisk doesn't support GPT. Use GNU Parted.
...

Let's create for example ten primary partitions of 10 MB each through a simple bash script.

root@ubuntu-server:~# j=1 ; for ((i=11; i<=101; i+=10)); do parted /dev/sdb mkpart primary $j $i; j=$i ; done

root@ubuntu-server:~# parted /dev/sdb print
Model: VMware, VMware Virtual S (scsi)
Disk /dev/sdb: 1074MB
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number  Start   End     Size    File system  Name     Flags
 1      1049kB  10.5MB  9437kB               primary
 2      10.5MB  21.0MB  10.5MB               primary
 3      21.0MB  31.5MB  10.5MB               primary
 4      31.5MB  40.9MB  9437kB               primary
 5      40.9MB  51.4MB  10.5MB               primary
 6      51.4MB  60.8MB  9437kB               primary
 7      60.8MB  71.3MB  10.5MB               primary
 8      71.3MB  80.7MB  9437kB               primary
 9      80.7MB  91.2MB  10.5MB               primary
10      91.2MB  101MB   9437kB               primary

If we take a look at the GPT table, we can distinguish the following parts.

root@ubuntu-server:~# dd if=/dev/sdb bs=512 count=4 | xxd -c 16
0000000: 0000 0000 0000 0000 0000 0000 0000 0000  ................
...
00001f0: 0000 0000 0000 0000 0000 0000 0000 55aa  ..............U.
0000200: 4546 4920 5041 5254 0000 0100 5c00 0000  EFI PART....\...
...
0000400: a2a0 d0eb e5b9 3344 87c0 68b6 b726 99c7  ......3D..h..&..
0000410: 4e6e ad36 2fec 8046 bc1f 4a42 82d2 8052  Nn.6/..F..JB...R
0000420: 0008 0000 0000 0000 ff4f 0000 0000 0000  .........O......
0000430: 0000 0000 0000 0000 7000 7200 6900 6d00  ........p.r.i.m.
0000440: 6100 7200 7900 0000 0000 0000 0000 0000  a.r.y...........
...

First up, it is the legacy MBR, which GPT holds for reasons of compatibility (the code 0x55AA points to the end of the MBR). The second part of the GPT (512 bytes) contains the header information for GUID partitioning. And the first partition entry appears in position 0x400.

GPT, beyond the MBR (I)

Have you ever wondered what are the limits of the MBR (Master Boot Record)? I mean, the maximum size of a partition or a entire hard drive able to be handled. The answer is 2.2 TB.

Below you can observe the structure of the MBR (a total size of 512 bytes, where each row is 32 bytes).

root@ubuntu-server:~# dd if=/dev/sda bs=512 count=1 | xxd -g 4 -c 32
0000000: eb639010 8ed0bc00 b0b80000 8ed88ec0 fbbe007c bf0006b9 0002f3a4 ea210600  .c.................|.........!..
0000020: 00bebe07 3804750b 83c61081 fefe0775 f3eb16b4 02b001bb 007cb280 8a74018b  ....8.u........u.........|...t..
0000040: 4c02cd13 ea007c00 00ebfe00 00000000 00000000 00000000 00000080 01000000  L.....|.........................
0000060: 00000000 fffa9090 f6c28074 05f6c270 7402b280 ea797c00 0031c08e d88ed0bc  ...........t...pt....y|..1......
0000080: 0020fba0 647c3cff 740288c2 52bb1704 80270374 06be887d e81701be 057cb441  . ..d|<.t...R....'.t...}.....|.A
00000a0: bbaa55cd 135a5272 3d81fb55 aa753783 e1017432 31c08944 04408844 ff894402  ..U..ZRr=..U.u7...t21..D.@.D..D.
00000c0: c7041000 668b1e5c 7c66895c 08668b1e 607c6689 5c0cc744 060070b4 42cd1372  ....f..\|f.\.f..`|f.\..D..p.B..r
00000e0: 05bb0070 eb76b408 cd13730d f6c2800f 84d000be 937de982 00660fb6 c68864ff  ...p.v....s..........}...f....d.
0000100: 40668944 040fb6d1 c1e20288 e888f440 8944080f b6c2c0e8 02668904 66a1607c  @f.D...........@.D.......f..f.`|
0000120: 6609c075 4e66a15c 7c6631d2 66f73488 d131d266 f774043b 44087d37 fec188c5  f..uNf.\|f1.f.4..1.f.t.;D.}7....
0000140: 30c0c1e8 0208c188 d05a88c6 bb00708e c331dbb8 0102cd13 721e8cc3 601eb900  0........Z....p..1......r...`...
0000160: 018edb31 f6bf0080 8ec6fcf3 a51f61ff 265a7cbe 8e7deb03 be9d7de8 3400bea2  ...1..........a.&Z|..}....}.4...
0000180: 7de82e00 cd18ebfe 47525542 20004765 6f6d0048 61726420 4469736b 00526561  }.......GRUB .Geom.Hard Disk.Rea
00001a0: 64002045 72726f72 0d0a00bb 0100b40e cd10ac3c 0075f4c3 ab9d0600 00008020  d. Error...........<.u......... 
00001c0: 2100831a 3b1f0008 00000098 0700003b 1b1f051f d00ffea7 07000250 b8000000  !...;..........;...........P....
00001e0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 000055aa  ..............................U.

The first 440 bytes contains the bootstrap code area, which takes care of starting up the operating system present on the active partition. Then, it is the disk signature, a 4-byte number that is randomly generated when the MBR is first created. It is an identifier which applies to the whole hard drive (not a single partition). After that, there are a couple of bytes set to null, and next, the partition table.

The partition table is made up of four entries of 16 bytes each which define the position and size of the sectors (LBA or Logical Block Addressing). In order to work around the problem of having just four partitions, these primary partitions can contain an arbitrary number of logical partitions. This schema can lead to problems, because some operating systems can only boot from primary partitions.

On the other hand, how do I work out the figure of 2.2 TB as top size for a partition? In conjunction with the universal sector size of 512 bytes and the 32-bit LBA pointers used by MBR partitions, you have ( 2^32 ) - 1 sectors * 512 bytes per sector.

(Also say that the MBR ends with the sequence of two bytes 0x55AA).

On account of the current size of hard drives and RAID technologies, the problem is really serious and will become more severe over time. In order to overcome it, the GUID partition table (GPT) is the natural successor to the MBR partition table.

GPT, supported on Linux since the 2.6.25 kernel version, uses 128-byte LBA modern tables, so therefore it is possible to have hard drives up to 8 ZB addressable for a disk with a 512-byte sector size. In addition, GPT can manage up to 128 partitions, so there is no need for extended or logical partitions. If you are interested, you can read up more about this topic on the Internet, since the aim of this article is to put forward how to manage this technology on Linux (also mention that for my tests, I used an Ubuntu Server 12.04 distribution).

First of all, point out that fdisk does not work with this partitioning scheme, but the good news is that can be got over with other Linux tools, such as parted. Keeping on with the tests, I have added a second hard drive to my system.

root@ubuntu-server:~# fdisk -l /dev/sdb 

Disk /dev/sdb: 1073 MB, 1073741824 bytes
255 heads, 63 sectors/track, 130 cylinders, total 2097152 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Disk /dev/sdb doesn't contain a valid partition table

Remote log server via HTTP (III)

Once finished the second article about the configuration of Samba, Remote log server via HTTP (II), now the server is going to be set up in order to be able to import the log directories via NFS and Samba. Furthermore, they will be served via Apache and backed up from time to time.

First of all, you have to create the directory where the logs will be mounted, as well as the backup directories.

[root@server ~]# mkdir -p /mnt/shared/logs 

[root@server ~]# mkdir -p /backup/logs/nfs /backup/logs/samba

So that SELinux allows Apache to access a directory brought by NFS or Samba, you have to enable the variables httpd_use_nfs and httpd_use_cifs. In addition, you have to change the SELinux security context of each directory imported.

[root@server ~]# setsebool -P httpd_use_nfs=on httpd_use_cifs=on

[root@server ~]# chcon -R -u system_u /mnt/shared/logs

[root@server ~]# chcon -R -t httpd_sys_content_t /mnt/shared/logs

Because the log server will not share any data through NFS and will offer no service by means of portmap, you will be able to deactivate the nfslock service.

[root@server ~]# service nfslock stop

[root@server ~]# chkconfig nfslock off

If you want to mount the NFS remote directory from client by hand, run the following order.

[root@server ~]# mount -t nfs -o soft,intr client.local:/var/log /mnt/shared/logs

And for the case of Samba.

[root@server ~]# mount -t cifs -o username=samba_logs,password=xxxxxx,soft //client.local/logs /mnt/shared/logs

The problem of mounting a remote directory statically is that the traffic passed down over the network is also increased, since when a file is updated, it is refreshed in the destination in the same way.

Moreover, you have to take into account another severe problem related to mount file systems via Samba, and is that if the connection is cut off (restarted, some network problem, etc.), Samba does not reconnect and the mount point can remain in an unstable state, thereby any existing synchronization would be lost. Thus, it is really important to always mount file systems by using automount.

Automount is a useful tool which takes care of mounting a directory when it is really accessed. It has got a timeout (600 sg by default) that when it is completed, the directory is automatically unmounted. This situation leads to reduce the network traffic (there may be long periods where you are not accessing the shared space) and avoid loss of synchronization. Also say that automount is managed by the autofs daemon.

This is the configuration used by automount to mount the log directory from client.

[root@server ~]# yum install autofs

[root@server ~]# vim /etc/auto.master
...
/mnt/shared/logs    /etc/auto.logs   -g,--timeout=300

[root@server ~]# cat /etc/auto.logs 
nfs    -fstype=nfs,soft,intr    client.local:/var/log
samba  -fstype=cifs,username=samba_logs,password=xxxxxx,iocharset=utf8,soft    ://client.local/logs

[root@server ~]# chmod 600 /etc/auto.logs

[root@server ~]# service autofs restart

The soft option is used for an application which is trying to access the shared area does not keep blocked if the connection is lost, and brings the control back to the system after 0.7 sg. With intr, allows the user to send an interruption signal if the application which uses NFS hangs.

If instead of hooking up to a Linux machine via Samba you have a Windows machine inside a domain, you would have to specify the domain name through the domain parameter.

Also mention that when you are mounting a directory from a Windows server, it might happen that strange characters turn up. This is due to the character conversion. So as to fix it, you have to use the iocharset=utf8 option for each mount point.

Remote log server via HTTP (II)

Let's keep on with the second article about setting up a Remote log server via HTTP. In the preceding part, the NFS daemon was configured in order to be able to export the local log server through NFS, and all this correctly secured by iptables and TCP wrappers. In this article, I am going to continue with the configuration of Samba.

First up, a new user called samba_logs will be adding to the system. From this user, the server machine will be able to hook up to the log directory via Samba. This user will not have neither a personal directory within home nor a shell.

[root@client ~]# useradd -d /dev/null -s /sbin/nologin samba_logs

In turn, this user will also be used to create an ACL (Access Control List) on the /var/log directory, granting read permissions to that user.

[root@client ~]# setfacl -R -m d:u:samba_logs:r /var/log/

[root@client ~]# getfacl /var/log/
...
default:user:samba_logs:r--
...

Then the samba package will be installed and configured.

[root@client ~]# yum install samba

[root@client ~]# cat /etc/samba/smb.conf
[global]
...
     hosts allow = 192.168.1.
...
[logs]
     comment = Log directory
     path = /var/log
     read only = yes
     valid users = samba_logs

Finally, the samba service will be restarted and marked as persistent. Furthermore, the user will be added to the local smbpasswd file.

[root@client ~]# service smb restart

[root@client ~]# chkconfig smb on

[root@client ~]# smbpasswd -a samba_logs

So as to shield the server by iptables, the following rules will be set into the /etc/sysconfig/iptables file (Samba uses the ports 137, 138 and 139 TCP/UDP).

[root@client ~]# cat /etc/sysconfig/iptables
...
-A RH-Firewall-1-INPUT -s server.local -p tcp --dport 137:139 -j ACCEPT
-A RH-Firewall-1-INPUT -s server.local -p udp --dport 137:139 -j ACCEPT
...

[root@client ~]# service iptables restart

Remember that is important to keep SELinux and TCP wrappes on. In order SELinux to let read the exported files, it is necessary to activate the variable samba_export_all_ro.

[root@client ~]# getenforce
Enforcing

[root@client ~]# setsebool -P samba_export_all_ro on

And below you can observe the configuration for iptables.

[root@client ~]# cat /etc/sysconfig/iptables
...
-A RH-Firewall-1-INPUT -s server.local -p tcp --dport 137:139 -j ACCEPT
-A RH-Firewall-1-INPUT -s server.local -p udp --dport 137:139 -j ACCEPT
-A RH-Firewall-1-INPUT -s server.local -p tcp --dport 445 -j ACCEPT
...

Now we can try out that everything is properly configured by running the next command on server.

[root@server ~]# yum install samba-client cifs-utils

[root@server ~]# smbclient -U samba_logs -L client.local
Enter samba_logs's password: 
Domain=[MYGROUP] OS=[Unix] Server=[Samba 3.5.10-125.el6]

    Sharename       Type      Comment
    ---------       ----      -------
    logs            Disk      Log directory
    IPC$            IPC       IPC Service (Samba Server Version 3.5.10-125.el6)
    samba_logs      Disk      Home Directories
Domain=[MYGROUP] OS=[Unix] Server=[Samba 3.5.10-125.el6]

    Server               Comment
    ---------            -------

    Workgroup            Master
    ---------            -------