Encryption support is one of the long-awaited features in Zabbix. There were various proposals to provide it: from pre-shared secret key (PSK) authentication to full TLS and Kerberos support. A year ago it was decided to go ahead and provide encryption support based on TLS.
Continue reading “Enable Safe Data Transmissions with Zabbix 3.0 Encryption Support”
Until Zabbix 2.0.7 proxies used a simple algorithm for their configuration synchronization – overwriting the old configuration with a new one. Even unchanged records were updated. It produced a lot of SQL updates even when there were no changes in configuration.
It worked good enough until one user wanted to synchronize a heavy loaded proxy (800 hosts, 120 000 items, 2000 NVPS) configuration every 7 minutes (see the graph).
The synchronization was taking about 2-3 minutes, produced more than 5000 updates per second and caused a heavy impact on performance of other Zabbix processes by blocking the hosts table. During this time poller processes could not update host status data.
In Zabbix 2.0.7 the synchronization aims to generate a minimum number of SQL queries by doing more work in the Zabbix proxy itself. The old configuration is read into memory and compared with the new configuration, record by record. If corresponding records exist in both old and new configurations every field is compared to find out differences. If all fields match, no update is necessary. If a few fields differ an SQL update statement is composed only for those fields.
Tests show that a configuration with 500 hosts and 600 000 items can be synchronized in ~11 seconds if there are few or no changes (not counting ~14 sec for getting 63 MBytes of configuration data from master server). That is 5 times faster than before (depends on the number of items).
After upgrading to Zabbix 2.0.7 the synchronization produces no updates if configuration is not changed as shown in the graph. The main improvement is less blocking of tables and shorter waiting times for other processes.
This came as the first assignment for me, a new Zabbix employee – to fix a bug ZBX-3788 zabbix daemon processes hang on futex
Sometimes terminating a Zabbix agent with “killall -15 zabbix_agentd” resulted in one hanging process and other processes in the <defunct> state. strace utility showed the process was hanging on futex:
Process 10468 attached - interrupt to quit
futex(0xb783ca50, FUTEX_WAIT_PRIVATE, 2, NULL
Continue reading “Mysterious Zabbix Problems: How We Debug Them”