Oct 28, 2012

Zabbix poller processes more than 75% busy and queue delay (II)

After putting forward the issues turned up on my current Zabbix installation and related to its performance (Zabbix poller processes more than 75% busy and queue delay), I am going to explain to you how I solved it.

First of all, I tried out to increase the number of pre-forked instances of pollers for the Zabbix server, that is, I changed its default value from 5 to 256 (remember that for that case, you have to set the the number of maximum connections in MySQL - max_connections - higher than 256, since every single poller opens a dedicated connection to the database).

root@zabbix-server:~# cat /etc/zabbix/zabbix_server.conf
...
# StartPollers=5
StartPollers=256

root@zabbix-server:~# cat /etc/mysql/my.cnf
...
max_connextions = 512

Below you can see the outcome after applying it (Zabbix server performance).




And the Zabbix data gathering process.




In the first figure, you can observe that the Zabbix queue has gone from 48 to 30 (approximately), and for the second one, the Zabbix busy poller processes went from 100% to 24%. So it is clear that if you have a server with enough resources, there is no problem to start many pollers. These kind of processes are responsible for requesting the data defined in the items, so the more pollers have available, the less overloaded the system is.

Other Zabbix server parameter that you ought to take into account is for example the Timeout (specifies how log pollers wait for agent responses). Try not to assign a very high value. Otherwise, the system might get overloaded.

Next week, I will end up this series of articles by accomplishing the part of the client.


15 comments:

  1. Great tips!!, I got a big performance gain :

    https://dl.dropbox.com/u/7405217/zabbix_performance.png

    https://dl.dropbox.com/u/7405217/zabbix_performance2.png

    thanks man!!

    ReplyDelete
  2. hello i have zabbix2.0.3 and i cant find the line max_connextions = 512 in zabbix_server.conf

    ReplyDelete
    Replies
    1. That parameter does not belong to the Zabbix configuration. It is located in the my.cnf file (MySQL).

      Delete
  3. should be max_connections

    ReplyDelete
  4. This comment has been removed by the author.

    ReplyDelete
  5. O meu não deu certo, estou usando o postgres, alterei o max_connections for 1000 e o StartPollers for 500, and problem continuos

    ReplyDelete
  6. Good afternoon,

    we experience since today that Zabbix busy poller processes is always above 80%

    i've changed startpollers and max_connections

    ReplyDelete
  7. This procedure solved my problem.
    Thank you for the post.

    Regards.

    Carlos (São Paulo, Brazil)

    ReplyDelete
  8. Hi,

    I have a doubt here. I also increased my pollers but how can i guess the approx number to which i should increase? 256 in your case . How did you come up with this number?

    ReplyDelete
  9. This comment has been removed by the author.

    ReplyDelete
  10. ...
    # StartPollers=5
    StartPollers=256

    => Results: PROBLEM: Too many processes on Zabbix

    ReplyDelete
    Replies
    1. modify the trigger for that to a higher number.....

      Delete
  11. I also face the same issue. I set the limit for StartPllers=256 but service is not restarting then i reduce it to 50 and everything working fine.

    Now can you please tell me how we can find the exact value we can set for StartPllers and how we can find it?

    Thanks

    ReplyDelete