SNMP is one of the most used monitoring types inside Zabbix. But it is also one of the most frustrating tools to configure for newcomers. Even experienced guys tend to make simple mistakes during configuration, which later lead to a lot of issues with flapping items, interfaces, false-positive notifications, or even worse.
Watch the video now.
In this post, I will talk about the things to remember and follow to successfully monitor devices through SNMP protocol within your installations. I will try to explain key concepts about SNMP, and the most important things that you need to keep in mind while setting up your first SNMP item.
SNMP — Simple Network Management Protocol, which works by query and is mostly used to monitor network devices such as switches, routers, some firewalls, or any storages.
If you click on Items section of any host in Configuration > Hosts and open Item parameters, for Type you can always choose between SNMPv1 agent, SNMPv2 agent, SNMPv3 agent, or SNMP trap.
The differences between SNMPv1, SNMPv2, and SNMPv3 are functional. The main distinctions:
- SNMPv1, which is the oldest version is not secure as it’s not encrypted. The only thing that is used to confirm communication between the monitoring host and a device is the community. SNMP community is configured on the device and then used by the agent to get a metric. In addition, SNMPv1 is limited only to 32-bit counters and is outdated.
- SNMPv2 is not encrypted as well and uses only the community for identification (community name). SNMPv2 is able to use 64-bit counters.
- SNMPv3 allows encrypting communication between the network device and Zabbix server or Zabbix proxy, as well as for proper identification with the username and a password.
SNMPv3 is much more secure but it is also much more complicated to maintain as you need to create the users and set a lot of parameters, such as Context name, Security name, Security level, etc. You also need to maintain this configuration in all monitored environments and devices.
The most frustrating thing with SNMP is that Key does not mean anything, it’s just a placeholder. You won’t find any key that will collect network traffic from a device. Here you need to use is SNMP OID — an object identifier which is a string of numbers, identifying monitored devices and their statuses.
SNMP OIDs link to the specific object in the MIB (Management Information Base), which can be specific for each device. These are the unique identifiers linking to the metric.
So, OID is what you need to configure on your item to receive data from SNMP items.
Usually, the problem is that you don’t know the OID. You only know that you need to monitor some devices and you don’t know those OIDs for the metrics that you need to monitor.
In this case, the easiest option is to use the MIB file for the needed network device that will contain information about all available metrics and ways to collect them. You can find many browser applications, which can open and visualize MIB files of the device. But there’s no way to import a MIB file into Zabbix.
NOTE. Zabbix is able to monitor SNMP checks but it does not connect directly to outside network devices. Zabbix is using snmpd on your machine. So, when Zabbix queries an item it asks SNMP, and snmpd connects to the device and requests the data.
New MIB files should be added to the default MIB files location (at least, in most Linux systems) which you can open in the CLI:
If you make some changes to a MIB file, you need to load them to SNMP daemon. The ways to do that are available in the documentation. For instance, you can use snmptranslate command to get a MIB name of an object based on its OID.
1. So, to add a new MIB file you need to edit your SNMP configuration file. If you don’t know where it is, you can use net-snmp-config –snmpconfpath command and add mibs + and a MIB name.
2. It is also possible to add absolutely all the MIB files that you have inside the MIB path:
In that case, you need to add snmp.conf file:
Despite adding absolutely everything in the directory, after changes are made, you need to restart both SNMP daemon and Zabbix.
When configuring the host, it’s not possible to add any SNMP monitoring item unless you address SNMP interface. To start monitoring SNMP, you need to click Add against SNMP interface, specify IP address of device that you will be monitoring or DNS name and a port of your SNMP interface which by default is ‘161‘.
Use bulk requests is turned on by default. But if you see items flapping and changing their status constantly, or SNMP interface jumping from green to red, from available to not available status, the only solution to fix monitoring might be to disable bulk requests.
NOTE. When monitoring is performed, Zabbix doesn’t perform a separate connection to your network device to poll every metric or every dataset. Instead, Zabbix requests data in the bulk. If you have bulk requests enabled, Zabbix makes the first connection to get a single metric, then pauses. If everything is fine, next time Zabbix collects two metrics, then three, and then continues with every next value multiplied by 1.5 . Up to 128 connections are performed only. But some devices are not able to respond to the requests in a bulk. In this case, some requests will not be supported at every connection.
NOTE. Don’t forget to reload the configuration cache and the proxies (if you have them).
SNMPv3 configuration mistakes
1. SNMPv3 is more secure but is slower because of encryption and authentication. When defining many item parameters for SNMPv3 you should be careful not to make typos. Otherwise, some items will not be supported on your hosts, or you will have SNMP interface flapping between red and green..
2. The second most popular mistake with SNMPv3 is related to snmpEngineID. If you’re monitoring SNMPv3 devices, snmpEngineID, which is configured on the device itself, must be unique.
NOTE. “Within an administrative domain, an snmpEngineID is the unique and unambitious identifier of SNMP engine. Since there is a one-to-one association between SNMP engines and SNMP entities, it is also unique and unambiguously identifies the SNMP entity within that administrative domain.”
As far as I know, by default engineID is the device MAC address, which should be unique. But network administrators tend to change these across their environment. So, if you have to monitor devices with exactly the same engineID you will have issues. Again, the easiest way to fix this is to check SNMP interface bulk requests, and then — snmpEngineID of your devices.
There are several options to retrieve snmpEngineID:
1. As snmpEngineID is also an object, it has object identifier, so you can use snmpget to retrieve this metric from your device and compare it with some other devices.
2. You can just physically check the configuration of the monitored device.
3. The most complicated way is to collect tcpdump to check communication on the line. Since you use SNMPv3, you have to specify credentials and identification. Then you would be able to decrypt it and check if snmpEngineIDs are actually unique.
When you create some new items or templates for your network monitoring and you don’t know OIDs that you need to monitor or MIB files, I suggest that you try to do everything on your own instead of just searching for a template that somebody already made.
If you need the most generic metrics as incoming traffic, outgoing traffic, or interface discards, default SNMP templates may be used as OIDs of generic metrics are usually the same across multiple vendors.
If you’re want to monitor something really specific for your device then just try to search, for instance, for Cisco OIDs. But before creating an item and adding it to the template which will be linked to thousands or hundreds of your Cisco devices, run snmpget in the CLI and query the needed value from your device. Add new items and create new templates only if you are sure they are correct.
That’s all for today on SNMP configuration. You are welcome to like, comment, and subscribe.