Sizing up sockets in Python

Last month I have to face the following issue at work related to sockets. We needed some control servers in order to use them in our test environment, but with the peculiarity that they had to be really fast, to be able to serve thousands of requests per second.

The first thought is that perhaps, this kind of server will have to be developed in C, but on the other hand, to perform a quick test in order to measure its behaviour in Python does not take long. So in this way, I made up a couple of Python scripts (client and server) so as to check it out. Let's see first of all, the part of the server.

from socket import socket, AF_INET, SOCK_STREAM
import datetime
import time

PORT = 22222
PACKET_SIZE = 1024
DELAY = 0
MAX_RETRIES = 10


def server():
    sock = socket(AF_INET, SOCK_STREAM)
    sock.bind(('', PORT))
    sock.listen(1)
    conn, _ = sock.accept()
    
    retries = 0
    data_list = []    
    
    n1=datetime.datetime.now()
    while True:
        data = conn.recv(PACKET_SIZE)
        time.sleep(DELAY)
        if data:
            data_list.append(data)
        else:
            retries += 1
            if retries == MAX_RETRIES:
                n2=datetime.datetime.now()
                total_time = str(n2-n1).split(":")[2]
                
                print "TOTAL TIME (sg): " + str(total_time)
                print "NUMBER OF PACKETS RECEIVED: " + str(len(data_list))
                print "RATE (pkts/sg): " + str(len(data_list) / float(total_time))
                
                break


if __name__ == "__main__":
    
    server()

As you can observe above, the server receives packets of a fix size, in this case 1 KB, and each packet is processed (inserted into a list) without any type of delay. When the input buffer is empty (the program has reached the maximum number of retries), the script will show a little summary related to the test (total time, number of received packets and rate). Also point out that the execution time is metered by means of datetime objects.

Let's take a look at now the side of the client.

from socket import socket, AF_INET, SOCK_STREAM

PORT = 22222
PACKET_SIZE = 1024
PACKETS_NUMBER = 10000


if __name__ == "__main__":

    padding = ""
    for i in range(PACKET_SIZE):
        padding += str(1)

    sock = socket(AF_INET, SOCK_STREAM)
    sock.connect(("172.16.75.132", PORT))

    for i in range(PACKETS_NUMBER):
        sock.send(padding)

The client script just sends a fix number of packets (10.000 in my case) with a concrete size (1 KB).

For the test, I am going to run the server on a virtual machine with Ubuntu Server 12.10 installed on it, and the client will be executed on my personal PC against the aforementioned server. This is the output from the server.

$ python server.py 
TOTAL TIME (sg): 00.138241
NUMBER OF PACKETS RECEIVED: 10005
RATE (pkts/sg): 72373.6084085

As you can see, this is an excellent result. More than 70.000 packets have been received and inserted into a list by the server, in spite of being working with sockets in Python. This result came out more than sufficient for our purposes and allowed us to be able to put into practice our required servers in Python.

Updating the key name in a Python dictionary

This is going to be a short text just to remind me how to modify quickly the name of a key in a Python dictionary. It is straightforward operation that I would like to comment. The idea consists in popping the key and its value, and then, putting it into the dictionary again with a new name.

Let's see an example. The following dictionary has a couple of keys, protocol and size, and I want to change the "protocol" key to "inet_protocol", and in addition, preserve its content.

>>> my_dict = {"protocol": "HTTP", "size": 1024}
>>> my_dict["inet_protocol"] = my_dict.pop("protocol")
>>>
>>> print my_dict
{'inet_protocol': 'HTTP', 'size': 1024}