Low Level Discovery for Windows Services

Introduction

When monitoring Windows servers and services it would be really handy to have a check to see if all the services that are registered with the system to automatically start up during boot, are also started.

You can use the item services[automatic] and compare its results with the output of the item services[automatic,started] using a trigger. However, this is not a very nice approach to this problem because it will tell you that there is a difference between these two but it will not tell you exactly which service has gone down.

Low Level Discovery (LLD) could solve this problem, but unfortunately the Zabbix agent currently doesn’t handle service discovery. (Please see ZBXNEXT-1368 for a feature request to implement this in the agent and vote on it if you need a solution for this!)

So, how to solve the problem in the meanwhile?

Fortunately, LLD just uses the concept of item keys and values as its basis. When requesting an LLD discovery item (like vfs.fs.discovery or net.if.discovery), the agent prepares a JSON response that contains all the needed information for the server and sends it back as a item value.

This means that we can use a UserParameter to feed LLD JSON output to the Zabbix server so it can set up the items and triggers for us!

To set this up, we need a script of some sorts to generate the output and a template to provide the discovery rules and prototypes needed.

You can find both of these on github.

Agent Configuration

Save the ‘servdisc.ps1′ on the harddrive of the machine you would like to perform the discovery on and add it as a UserParameter in the zabbix_agentd.conf file:

UserParameter=service.discovery,powershell -File c:\servdisc.ps1

When testing this UserParameter, I found that the Powershell script was taking longer to execute then the default timeout value for the Zabbix agent (3 seconds). To work around this, increase the Timeout parameter in the agent configuration:

Timeout=10

Lastly, the script is not signed. This means that a default Windows install will not allow the script to run. To be able to run the script it should either be signed or the verification of scripts should be disabled. For this test I disabled the verification. Please verify with your internal security team if it is ok to do so in your environment as well.

Open a Powershell by going to ‘Start -> Run’ and typing in ‘powershell’ and hit enter. Then type this command to prevent verification of scripts:

set-executionpolicy Unrestricted

You can verify if the change was a success by running:

get-executionpolicy

You can then exit the Powershell.

Configuring a Host for Discovery

Import the ‘Template_Windows_Service_Discovery.xml’ template into your Zabbix frontend and link it to the host that you’ve installed the script on.

After a while, the discovery should find all the services in the system and Zabbix will create the appropriate triggers and items for them:

And, after a while, you can find the current service states under ‘Latest Data’ as well:

Conclusion

Even though Windows Services discovery is currently not implemented in the Zabbix agent, Zabbix does provide us with the tools to create our own solution to this problem.

Using scripts such as these show us the potential that Low Level Discovery has to make our lives easier when it comes to monitoring. If you have any ideas on how Low Level Discovery can be improved to solve your specific monitoring needs, please create a feature request for it in the support system.

In the meanwhile, you can build your own discovery checks when needed with the techniques outlined above.

This entry was posted in Technical and tagged , , , . Bookmark the permalink.

12 Responses to Low Level Discovery for Windows Services

  1. Oleksiy Zagorskyi says:

    If we increase agent’s Timeout then we need to increase server’s Timeout as well, which should be done carefully and with complete understanding how this parameter is important at the server side :)

    • Raymond Kuiper says:

      It is a proof of concept afterall :-) For a proper solution to this problem, vote on the ticket!

  2. Felix says:

    No import no item template, it takes a lot to make the discovery?

  3. HenrikJ says:

    Raymond, would you accept a pull request for a VB version of servdisc.ps1 ? Most our Windows 2003 boxes do currently not have Powershell installed …

    • Raymond Kuiper says:

      If by that you mean you would like to provide me with a vb version, be my guest (I’m still a bit new on the github front ;-) ) I’ve send you an email to discuss things further.

  4. HenrikJ says:

    Sure, I have a VB prototype done already. Let me finish that and I’ll ping you in IRC

  5. You can make Powershell run Base64 encoded code directly from the command line, that alleviates the need for a the execution policy change and the script on the hard drive. I did it like this:

    UserParameter=windows.services,powershell -NoLogo -NonInteractive -EncodedCommand dwByAGkAdABlAC0AaABvAHMAdAAgACIAewAgAGAAIgBkAGEAdABhAGAAIgA6AFsAIgANAAoARwBlAHQALQBXAG0AaQBPAGIAagBlAGMAdAAgAHcAaQBuADMAMgBfAHMAZQByAHYAaQBjAGUAIAB8ACAAVwBoAGUAcgBlAC0ATwBiAGoAZQBjAHQAIAB7ACQAXwAuAFMAdABhAHIAdABNAG8AZABlACAALQBlAHEAIAAnAEEAdQB0AG8AJwB9ACAAfAAgAEYAbwByAEUAYQBjAGgALQBPAGIAagBlAGMAdAAgAHsADQAKACAAIAAgACAAJABsAGkAbgBlACAAPQAgACAAIgAgAHsAIABgACIAewAjAFMAVABBAFQAVQBTAH0AYAAiADoAYAAiACIAIAArACAAJABfAC4AUwB0AGEAdABlACAAKwAgACIAYAAiACAALAAgAGAAIgB7ACMARABJAFMAUABMAEEAWQBOAEEATQBFAH0AYAAiADoAYAAiACIAIAArACAAJABfAC4ARABpAHMAcABsAGEAeQBOAGEAbQBlACAAKwAgACIAYAAiACAALAAgAGAAIgB7ACMAUwBFAFIAVgBJAEMARQBOAEEATQBFAH0AYAAiADoAYAAiACIAIAArACAAJABfAC4ATgBhAG0AZQAgACsAIAAiAGAAIgAgAH0ALAAiAA0ACgAgACAAIAAgAHcAcgBpAHQAZQAtAGgAbwBzAHQAIAAkAGwAaQBuAGUADQAKAH0ADQAKAHcAcgBpAHQAZQAtAGgAbwBzAHQAIAAiACAAXQAiAA0ACgB3AHIAaQB0AGUALQBoAG8AcwB0ACAAIgB9ACIA

    (Note that this script doesn’t use the same format in this blog as I wanted to shorten it. You can decode it at http://www5.rptea.com/base64/ and select UTF-16. All Powershell Base64 encoded scripts must be encoded in UTF-16)

  6. Theo says:

    Value should be a JSON object

    • Theo says:

      Some way to make it work without having to change the rule in powershell?
      is it why is appearing on the front end the error: Value Should be a JSON object

  7. BLinz says:

    Hi Rymond,

    great Work – but i get also the “Value should be a JSON object” Error.

    The Problem is that the JSON Output of your Script or the vbs Version ist to Long.
    I make a new Version in AutoIt so the servdisc script is now an exe-file.

    >ou can Download an Archiv with the servdisc.exe, the Original Template of Raymond Kuiper and a german Version of the Template at:
    http://znil.net/index.php?title=Zabbix:Template_Windows_Service_Auto-Discover_-_Windows_Dienste_automatisch_entdecken
    The Download Link ist at the top of the Article, the page is in german but the description and HowTo in the Archiv is in english.

    Thanks at Raymond for the Idea and the Template,

    Bernhard Linz

    • BLinz says:

      Ahh,

      Just Bullshit – if i only send 4000 Chars the List og Services is not complete everytime – and zabbix wanna delete the not found Services and also delete the item.
      I Change my Version so the Service Description will not used.

      You can use also the PowerShell or VBS Version – but Change the Script:

      PowerShell:
      Search for
      $objItem.Description
      Change to
      ” – ”

      VBS:
      Search for
      objService.Description
      Replace with
      ” – ”

      My JSON String have now up to 16500 Chars – but Zabbix 2.2 with MySQL Database can take this. Got 117 running Services with one Discovery.

      My Download Link in the Post before contains a new Version wich don’t use the Service Description.

      Bernhard