Today we are talking about a use case when it’s impossible to find a proper way to write a recovery expression for the Zabbix trigger. In other words, we know how to identify problems. But there is no good way to detect when the problem is gone.
This mostly relates to a huge environment, for example:
- Got one log file. There are hundreds of patterns inside. We respect all of them. We need them
- SNMP trap item (snmptrap.fallback) with different patterns being written inside
In these situations, the trigger is most likely configured to “Event generation mode: Multiple.” This practically means: when a “problematic metric” hits the instance, it will open +1 additional problem.
Goal:
I just need to receive an email about the record, then close the event.
As a workaround (let’s call it a solution here), we can define an action which will:
- contact an API endpoint
- manually acknowledge the event and close it
The biggest reason why this functionality is possible is that: when an event hits the action, the operation actually knows the event ID of the problem. The macro {EVENT.ID} saves the day.
To solve the problem, we need to install API characteristics at the global level:
{$Z_API_PHP}=http://127.0.0.1/api_jsonrpc.php
{$Z_API_USER}=api
{$Z_API_PASSWORD}=zabbix
NOTE
‘http://127.0.0.1/api_jsonrpc.php’ means the frontend server runs on the same server as systemd:zabbix-server. If it is not the case, we need to plot a front-end address of Zabbix GUI + add ‘api_jsonrpc.php’.
We will have 2 actions. The first one will deliver a notification to email:
After 1 minute, a second action will close the event:
This is a full bash snippet we must put inside. No need to change anything. It works with copy and paste:
URL={$Z_API_PHP}
USER={$Z_API_USER}
PASSWORD={$Z_API_PASSWORD}
# authorization
AUTH=$(curl -sk -X POST -H "Content-Type: application/json" -d '
{
"jsonrpc": "2.0",
"method": "user.login",
"params": {
"user": "'"$USER"'",
"password": "'"$PASSWORD"'"
},
"id": 1,
"auth": null
}
' $URL | \
grep -E -o "([0-9a-f]{32,32})")
# acknowledge and close event
curl -sk -X POST -H "Content-Type: application/json" -d '
{
"jsonrpc": "2.0",
"method": "event.acknowledge",
"params": {
"eventids": "{EVENT.ID}",
"action": 1,
"message": "Problem resolved."
},
"auth": "'"$AUTH"'",
"id": 1
}' $URL
# close api key
curl -sk -X POST -H "Content-Type: application/json" -d '
{
"jsonrpc": "2.0",
"method": "user.logout",
"params": [],
"id": 1,
"auth": "'"$AUTH"'"
}
' $URL
For more information about best Immigration consultant, https://www.visaexperts.com/
This is just what I was looking for! It worked perfect for me (I just had to change the url to: http: //127.0.0.1/zabbix/api_jsonrpc.php in my case)
Thank you very much!
Hi!
This blog post is very useful; at least – it provides some workaround while ZBXNEXT-2452 is not implemented.
However, since this blog has been published, some new versions of Zabbix (5.4 and 6.0) was published also; so below are my comments regarding to the current LTS version (v6.0).
First of all, v5.4 introduced the API tokens (current link). So, this script could be simplified to avoid login and logout on every operation.
Second, API documentation contains 2 important notes:
So, user (used for authentication in this script) must have read/write permissions onto an appropriate hosts; and trigger itself must permit a manual closing.
Finally, since v5.4 scripts were moved onto global settings: initially you should define a script here (Administration -> Scripts), and only then you are able to choose a defined script during the Action configuration. Good news is that configuring scripts, all macro’s – both system macro (like {EVENT.ID}) and user macro (like {$Z_API_PHP}) could be inserted in the text of script (they will be substituted during script execution), so the script could be simplified once more.
So, finally, you can use the following steps:
Text of the final script:
# acknowledge and close event curl -sk -X POST -H "Content-Type: application/json" -d ' { "jsonrpc": "2.0", "method": "event.acknowledge", "params": { "eventids": "{EVENT.ID}", "action": 1, "message": "Problem auto-closed." }, "auth": "{$Z_API_TOKEN}", "id": 1 }' {$Z_API_PHP}Once more idea: as this script consists of a single command performing an HTTP/HTTPS request, it could be re-written as a webhook.
Pros:
Cons:
Details:
try { var req = new HttpRequest(), params = JSON.parse(value), eventid = params.eventid, token = params.token, url = params.url, body, resp; Zabbix.log(4, '[Closing problem] starts: eventid=' + eventid + ', token="' + token +'", url="' + url + '".'); //Validation if (typeof eventid !== 'string' || eventid.trim() === '') { throw 'The eventid must be defined in macro "{EVENT.ID}".'; } if (typeof token !== 'string' || token.trim() === '') { throw 'The token must be defined in macro "{$Z_API_TOKEN}".'; } if (typeof url !== 'string' || !(url.startsWith('http://') || url.startsWith('https://'))) { throw 'The URL (starting with "http://" or "https://") must be defined in macro "{$Z_API_PHP}".'; } if (typeof params.HTTPProxy === 'string' && params.HTTPProxy.trim() !== '') { req.SetProxy(params.HTTPProxy); } //Processing logic req.addHeader('Content-Type: application/json'); body = { 'jsonrpc': '2.0', 'method': 'event.acknowledge', 'params': { 'eventids': eventid.trim(), 'action': 1, 'message': 'Problem auto-closed.' }, 'auth': token.trim(), 'id': 1 }; resp = req.post(url.trim(), JSON.stringify(body) ); if (req.getStatus() != 200) { throw 'Response code: ' + req.getStatus(); } Zabbix.log(4, '[Closing problem] returned: ' + resp); resp = JSON.parse(resp); if ('error' in resp) { throw ('API error: ' + resp.error.message + '; ' + resp.error.data); } } catch (error) { Zabbix.log(3, '[Closing problem] cannot close event "' + eventid + '": ' + error); throw 'Cannot close event ' + eventid + ': ' + error; } return 'OK';