Analyze your incoming metrics and look for interruptions in continuously increasing or decreasing metrics with the monoinc and monodec history functions.
A continuous increase or decrease is the norm for metrics such as server uptime, time remaining until a task is executed, number of daily transactions, and many other such use cases. A software or hardware failure could impact these counters and we need to ensure that they are providing the data in an expected manner.
Use monoinc and monodec history functions and detect if value monotonicity is true or false:
- Detect monotonicity over a number of values or a time period
- Strict and weak modes of monotonicity detection
- Receive alerts if a metric is not monotonic
- The monotonicity check can be combined with other functions to create flexible problem generation logic
Check out the video to learn how to use the monoinc and monodec history functions
How to configure monoinc and monodec history functions:
- Identify the items for which you wish to detect monotonicity
- For this example, the system.uptime key is used
- Navigate to Configuration → Hosts → Your host → Triggers
- Press the Create trigger button
- Provide the trigger name and severity
- Press the Add button to add the trigger expression
- Select the item, the monoinc function, evaluation period, mode and result
- For this example, we will use the strict mode
- An example expression: monoinc(/Linux server/system.uptime,1h,”strict”)=0
- Simulate a problem by restarting the host
- Navigate to Monitoring → Problems
- Confirm that the problem has been generated
Tips and best practices
- The functions return 1 if all elements in the evaluation period continuously decrease or increase, 0 otherwise
- The default mode – weak, checks if every value is bigger/smaller or the same as the previous one
- The strict mode checks if every value has increased/decreased
- Relative and absolute time shifts can be used to analyze time periods for monotonicity
Can this expression be used to do the opposite of what’s described here, as in trigger an alert when a value that’s normally static begins to increment or decrement?
For example, I have an SNMP value from a UPS that reports the battery capacity in percent. Normally the value will be at 100%. If the value starts to decrement, that would indicate the battery is discharging and I would like an informational alert to trigger alerting of that fact. Would the following expression work to indicate the value has decremented over the last 5 minute period:
monodec(/Vertiv/ifBatteryCapacity,5m)=1
Or is there a better way to go about this?