Can you make sure your video streams are up with Zabbix? Of course, you can! By day, I am a monitoring technical lead in a global cyber security company. By night, I monitor my home with Zabbix & Grafana Labs and do some weird experiments with them. Welcome to my weekly blog about the project.
You might have a surveillance camera at home to record suspicious activities in your yard while you are away or so. Most of the time the cameras do work just fine but might require a hard reboot from time to time, for example, due to harsh weather, or not coming back after a network outage. A networked camera responding to ping does not 100% mean the camera is actually functional. I have seen our camera going black and refusing to connect to its stream even though it thinks it’s working just fine.
Zabbix to the rescue!
Table of Contents
Connecting to your camera
My post for this week is mostly to maybe give you a new approach for monitoring your cameras, not so much a functional solution as I’m still figuring out how to do this properly.
For example, I can connect to our camera via RTSP protocol and pass some credentials with it, so rtsp://myusername:[email protected]:443/myAddress
To figure out a connection address for your camera model, iSpyConnect has a nice camera database.
Playing the stream
To test if the video stream works, VLC and mplayer are good options; for visually verifying the stream works, try something like
mplayer ‘rtsp://myusername:[email protected]:443/myAddress’
or for those who like to use a GUI, in VLC, File –> Open Network –> enter your camera address.
For obvious reasons, I am not posting here an image from our camera. Anyway, trust me, this method should work if you have a compatible camera.
Let’s go next for the neat tricks part, which I’m still figuring out myself, too.
Making sure the stream works
To make sure the video stream is up and running, make your Zabbix server, Zabbix proxy, or a dedicated media server to continuously stream your video feed. For example:
mplayer -vo null ‘rtsp://myusername:[email protected]:443/myAddress’
The combination above would make mplayer play the stream with a null video driver; thus, the stream will be continuously played, but just with no visual video output generated. In other words, under perfect conditions, the mplayer process should be running on the server all the time. If anything goes wrong with the stream, mplayer quits itself, and the process goes away from the process list, too.
Using Zabbix to check the player status
Now that you have some server continuously playing the stream, it’s time to check the status with Zabbix.
From here, checking the stream status with Zabbix is simple, just
- create a new item to check if for example mplayer process is around with Zabbix Agent item type and proc.num[,mplayer] key and
- make your Zabbix alert about it if the number of mplayer processes is <1
Camera screenshots to your Zabbix user interface
Both mplayer and VLC can be controlled remotely, so here’s an idea I have not yet implemented but testing out.
If a motion sensor, either an external unit or a built-in, detects movement, make Zabbix send a command to the camera to record a screenshot of the camera stream, or possibly a short video. Then just make the script to save the photo or video in a directory that Zabbix can access and then show with its URL widget type.
mplayer has a slave mode for receiving commands from external programs, which might work together with a FIFO pipe.
Real-time video stream in your Zabbix user interface
At least VLC can transcode RTSP to HTTP stream in real-time, so in theory, then embedding the resulting stream to your Zabbix user interface should very much be doable with a short HTML file and Zabbix URL widget type. This one I did not yet even start to try out, though.
So, that’s all for this week’s blog post. I’m still building this thing out, but if you have successfully done something similar, please let me know!
I have been working at Forcepoint since 2014 and am a true fan of functional testing. — Janne Pikkarainen
This post was originally published on the author’s LinkedIn account.
Hi Janne,
Probably, it’s needed some Action for the trigger for trying to restart this process again after a problem occurred, isn’t it?
In our case it would be to power cycle the smart power socket into which the camera is connected to. However, with my Zabbix — Cozify integration I’m only reading data from Cozify, so poking the smart power socket automatically via Zabbix is not (yet) possible for me. 🙂 Maybe one day.
OK, Janne, I got it.
However, I spoke just about the monitoring part. The criteria for Zabbix to detect a problem with camera is: a disappearing of process from the process list. Nevertheless, after the problem recovering (camera has been rebooted by some way) – Zabbix should detect this event, i.e. the crashed process ‘mplayer’ also should be restarted?
Yep, it depends on how the mplayer would be configured to be run. If it’s supposed to be up by as a system service, or even something as simple as “while true; do mplayerstuffgoeshere; done”, then it would not be Zabbix which would try to poke mplayer back to life, but Zabbix could easily notice if mplayer is not up or its lifetime is very short. There’s so many ways to do this 🙂