Why do we recommend upgrading to the latest version? Zabbix 5.0 goes with improved stability, performance, and security, as well as provides technical support for the next five years. What’s important, Zabbix 5.0 supports TimescaleDB, which is designed to enhance performance and compresses the data.
Contents
I. Introduction (1:46:24)
II. Why TimescaleDB (1:46:42)
III. What to upgrade (1:55:58)
IV. How to upgrade (1:57:29)
1. Upgrading PostgreSQL (1:58:43)
2. Backup (2:00:33)
3. Start the upgrade (2:01:51)
4. Upgrade front-end (2:02:27)
5. Install TimescaleDB (2:04:16)
6. Prepare your DB for TimescaleDB (2:05:34)
7. Check housekeeping settings (2:07:52)
V. Update results (2:08:41)
VI. Conclusion (2:11:11)
Introduction
Why upgrade your Zabbix instance? With newer versions you’ll get:
-
- Improved stability,
- Improved performance,
- Improved security,
- New feature support.
which are thoroughly described in our documentation and blog and most important support for the next five years.
Longer life cycle of newer versions
Why TimescaleDB
Zabbix 5.0 supports TimescaleDB — a PostgreSQL extension, which adds time-series based performance and data management optimizations to a regular PostgreSQL database. It enhances performance since 4.2. ,though in experimental way, and even compresses the data starting with 5.0, but now officially.
The main TimescaleDB architecture solutions are:
-
- Hypertables — abstraction layer and primary point of integration with your data used for creating tables and indexes, altering tables, inserting data, and selecting data.
- Chunks — the elements the hypertables are automatically split into, where each chunk corresponds to a specific time interval and a region of the partition key’s space.
TimescaleDB architecture solutions
And, even while looking at it from different angles, TimescaleDB looks like the safest bet for Zabbix and its users because:
-
- TimescaleDB is an extension, It doesn’t require extra hardware, virtual machines or any other infrastructure changes, you can continue to use your PostgreSQL tools of choice and SQL operations and queries.
- TimescaleDB lets you keep virtually all database related code in Zabbix intact.
- TimescaleDB ensures considerable performance improvements for Zabbix history syncer and housekeeper.
For instance, Vanilla PostgreSQL should be optimized as its values will eventually drop down.
PostgreSQL without TimescaleDB extension
TimescaleDB ensures consistency, and you get a lot more values per second inserted into your database in a timely manner and without sudden spikes.
PostgreSQL with TimescaleDB extension
A direct comparison shows that PostgreSQL eventually falls down, while Timescale DB keeps it steady with no big drops or no big jumps, just as it needs to be.
PostgreSQL with and without TimescaleDB comparison
Compression is a relatively new feature to TimescaleDB itself. TimescaleDB compression can reach up to 90%, which is quite a noticeable difference, so you can save a lot on the storage expenses.
TimescaleDB compression performance
How does TimescaleDB compression works? It basically turns all the tables into one single row collecting all the values. But this row is read-only, and updates are possible only after decompression.
TimescaleDB compression example
And to summarize all this information and why is it an interesting idea to at least give it a try:
-
- TimescaleDB is still SQL and it supports all SQL operations and queries.
- TimescaleDB is compatible with the existing PostgreSQL ecosystem and tooling. So, you’ll still be able to use your clusters and query execution.
- TimescaleDB is quite transparent in terms of time and space partitioning for both scaling up and on a single node.
- TimescaleDB is accessible for the newcomers as documentation on PostgreSQL and TimescaleDB is available.
What to upgrade
So eventually you’ve decided to upgrade your Zabbix and start using TimescaleDB, what will you need to upgrade?
-
- Zabbix server.
- Database. If you use some older PostgreSQL version, then you will need to update the database to at least PostgreSQL 9.6. If you want to use compression, then you need to upgrade to at least PostgreSQL 10.2.
- Web front-end.
- Proxy.
- Java Gateaway.
You don’t need to upgrade:
-
- Zabbix agent, as agents are still used, they are backward-compatible (though older agents don’t support new features).
- Database (if you already have newer versions).
How to upgrade
Upgrading Zabbix and especially databases can be quite stressful, and the question is always about what to do first.
For an average setup:
-
- CentOS 7
- PostgreSQL 9.6
- Zabbix server 4.0
- Your average hardware
You need to upgrade your DB to at least PostgreSQL version 10.2 or higher to use TimescaleDB compression. Or you can upgrade to the latest PostgreSQL 12.
Upgrading PostgreSQL
1. Install the repo RPM for the version you will be using:
yum install https://download.posgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm
2. Install new PostgreSQL version.
yum install postgresq12-server
3. Stop the current PostgreSQL version (don’t forget the server).
systemctl stop postgresql-9.6
4. Initialize the new version.
sudo su postgres cd ~/ /usr/pgsql-12/bin/initdb -D /var/lib/pgsql/12/data/
5. Migrate the data.
/usr/pgsql-12/bin/pg upgrade --old-datadir /var/lib/pgsql/9.6/data/ --new-datadir /var/lib/pgsql/12/data/ --old-bindir/usr/pgsql-9.6/bin/ --new-bindir /usr/pgsql-12/bin/
6. Start the new instance.
systemctl start postgresql-12
NOTE. The migration of the data will take time if your database is already enormous, so plan your time accordingly.
Backup
Now you need to back up, though it’s a good idea to actually back up everything before migration.
-
- Read the release notes and take note of the important changes at https://www.zabbix.com/documentation/5.0/manual/installation/upgrade/packages/rhel_centos.
- Make Zabbix database backup.
pg_dump dbname > dbname.bak
You can also back up only the configuration files if you don’t need history information and you don’t want to start completely from scratch losing all the data. Since this will allow you yo keep all your server configurations, scripts, or some branding or customizations in the front-end.
Configuration files backup
Start the upgrade
1. Stop Zabbix server to make sure that no new data is inserted into database:
systemctl stop zabbix-server
2. Upgrade your current repository package:
rpm -Uvh https://repo.zabbix.com/zabbix/5.0/rhel/7/x86_64/zabbix-release-5.0-1.el7.noarch.rpm
3. Upgrade Zabbix components:
yum upgrade zabbix-server-pgsql zabbix-agent
Upgrade front-end
NOTE. If you have any customizations, back up the front-end configuration files.
1. Remove old front-end:
yum remove zabbix-web-*
2. Install SCL repository (to be able to use both PHP versions):
yum install centos-release-scl
3. Edit /etc/yum.repos.d/zabbix.repo file:
[zabbix-frontend]name=Zabbix Official Repository frontend - $basearchbaseurl=http://repo.zabbix.com/zabbix/5.0/rhel/7/$basearch/frontendenabled=1
4. Install new front-end packages:
yum install zabbix-web-pgsql-scl
5. Update timezone in /etc/opt/rh/rh php72/php fpm.d zabbix.conf file.
6. Start and enable php-fpm service:
systemstl start rh-php72-php-fpm
7. Restart Apache:
systemctl restart httpd
Install TimescaleDB
1. Add TimescaleDB repo:
https://docs.timescale.com/latest/getting started/installation/rhel centos/installation yum
TimescaleDB repo
2. Install appropriate package for PG version
yum install -y timescaledb-postgresql-12
3. Configure your PostgreSQL:
timescaledb-tune --pg-config=/usr/pgsql-12/bin/pg_config
NOTE. When tuning your database don’t blindly agree with everything, as tuning may interfere with how your Zabbix instance performs or PostgreSQL overall.
Prepare your DB for TimescaleDB
1. Start Zabbix server to update the DB schema.
2. Stop Zabbix server.
3. Enable TimescaleDB extension:
echo "CREATE EXTENSION IF NOT EXISTS timescaledb CASCADE;" | sudo -u postgres psql zabbix
4. Run the timescaledb.sql script located in database/ postgresql:
cd /usr/share/doc/zabbix-server-pgsql-5.0.0/ cat timescaledb.sql | sudo -u zabbix psql zabbix
5. Wait — migration of existing history and trend data may take a lot of time. Zabbix server and front-end must be down for the period of migration.
Migration output
If you forget to start the Zabbix before starting to convert tables to hypertables, you will see some of them fail. In this case, you need to update this schema and run the script once again.
Migration mistakes
6. Start Zabbix server.
Check housekeeping settings
You need to check the housekeeper settings, as the conversion changes the history data, and the way it’s kept. Since you have enabled compression, which is not enabled by default, you will see a new line of setting in the front-end – compress records older than seven days.
Housekeeping settings
NOTE. 7 days — the minimum possible compression period, i.e. you cannot compress yesterday’s data.
Update results
The upgrade will boost your Zabbix performance by at least 20% without hardware changes, letting your process 2.7 K valuer per second instead of 2.4 K.
Performance before upgrade
Performance after upgrade
History syncer and housekeepers before the update can be quite loaded jumping occasionally up to 100%. percent.
Utilization before upgrade
With TimescaleDB, history sinker can basically take a vacation, as it was loaded up to 20% only, and the data is flowing in more efficiently as the database size is 90%-compressed.
Utilization after upgrade
In the upgrading examples, two databases were used — 30 GB and 300 GB.
Database size before upgrade
Database size after upgrade
Conclusion
-
- Compressed chunk modifications (inserts, deletes, updates) are not allowed.
- You can still use other databases, which do not use compression, but perform efficiently with partitioning, especially if you know how to operate with them better.
- Performance improvements up to 20-25%. without any hardware changes
- Save up to 90% percent of space by compressing the tables in TimescaleDB way
There are migration tools, like i.e. pgloader, which may help you to migrate your instance smoothly, but everything will depend on your setup.
See also: presentation video and slides.