Dec 22, 2010

Frag needed and DF set

I was remembering a curious problem that we had at work the last year.

There was an HTTPS service that when you tried to access it by means of a web browser, the screen did not show anything. The display was blank waiting...

The main inconvenience was that we had not access to the web server to check it out. At that moment, I ran a tcpdump on the client in order to capture all network traffic and try to find out what was happening.

When I analyzed the packets I could see that the TCP connection was established correctly but then, the web server was sending frames with "TCP Previous segment lost", "TCP Dup ACK", "TCP Retransmission" messages. It is a pity because I do not keep these network captures to output them here...

Well, then it was clear that there was some device in the middle of the route between client and server, which was causing a leak of network packets in that communication.

Our network architecture was similar to the schema of the following image.




Taking a look at the firewall logs, we could see that there were ICMP packets related to the problematic HTTPS connection which were being dropped.

The router was sending ICMP packets (type 3, destination unreachable - code 4, fragmentation needed) to the source, in order to warn it that its MTU was smaller than the size of the packets, and on top of all that, the DF (prohibit fragmentation) was set to 1.

When that ICMP packet reached to the firewall, it was dropped and the source never knew that it had to reduce the data field for the TCP/IP packets. The solution was to allow the ICMP traffic (ICMP protocol with type 3 and code 4).

We can realize a little test in our computer. For example, attempting to send a packet whose size exceeds our network MTU (1500) and besides, DF=1 (192.168.1.100 is my PC and 192.168.1.1 the destination).

javi@kubuntu:~$ ping -c 1 -s 2000 -M do 192.168.1.1 PING 192.168.1.1 (192.168.1.1) 2000(2028) bytes of data.
From 192.168.1.100 icmp_seq=1 Frag needed and DF set (mtu = 1500)

If I run a tcpdump, I can see that I receive an ICMP datagram noting me that I have to lower the size of the packets.

javi@kubuntu:~$ sudo tcpdump -ni lo icmp -s0
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on lo, link-type EN10MB (Ethernet), capture size 65535 bytes
18:59:33.014773 IP 192.168.1.100 > 192.168.1.100: ICMP 192.168.1.1 unreachable
- need to frag (mtu 1500), length 556


2 comments:

  1. Hola Javier,

    Al hilo del bit DF, ¿Es posible configurar en el linux que elimine el bit DF de los paquetes que vengan con ese bit habilitado?

    Un saludo,
    Luis

    ReplyDelete
    Replies
    1. Hola,

      Para deshabilitar que Linux marque un paquete con el bit DF activado creo recordar que tienes que configurarlo a través de uno de los parámetros del kernel: ip_no_pmtu_disc

      Por defecto está activado:
      #cat /proc/sys/net/ipv4/ip_no_pmtu_disc
      0

      Luego para deshabilitarlo esa variable tendría que valer 1. Para hacer el cambio permanente tienes que hacerlo a través del fichero sysctl.conf.

      La duda que tengo con respecto a esto es si afecta sólo a los paquetes con origen tu máquina linux, o también a los que pasan a través de ella (a modo de router - forwarding).

      Si en un forwarding te siguiera marcando los paquetes con el DF a 1 habiendo cambiado la variable que te he dicho, podrías utilizar la cadena mangle de iptables para modificar la cabecera del paquete IP añadiendo una regla a la tabla de POSTROUTING.

      Un saludo,

      Delete