Monitoring 18 servers during a live event isn’t an easy task, however there is a easy way to do create a dynamic page which will display you a status of desired servers using Nagios plugins, AJAX and a bit of PHP.
I’m running a cluster of 18 ec2 servers which are used for high capacity real time platform built by Monterosa (company which I work for), we are using Nagios and Munin for monitoring purposes however viewing all necessary data on one page in Nagios is impossible if you run that many servers and each of them has couple of monitors.
I’ve built a simple web page in PHP runing a loop which executes shell command running check_nrpe plugin from Nagios and printing output in browser window.
<?
$hosts = array("host1.abc.com", "host2.abc.com");
foreach ($hosts as $host)
{
$load = shell_exec("/usr/local/nagios/libexec/check_nrpe -H $host -c check_load -a 5.0,4.0,3.0 10.0,6.0,4.0 | cut -d '|' -f 1");
echo "$load;
}
?>
This is good for two servers but not if you have to get status of 18 as it will take ages before you will see anything on your screen.
I started to thinking that I could find some AJAX lib which will load content dynamically and I found this.
Quick implementation made a huge difference:
Status script
First we need a page which will get status of desired host defined by parameter in URL.
<?
$host = $_GET['host'];
$load = shell_exec("/usr/local/nagios/libexec/check_nrpe -H $host -c check_load -a 5.0,4.0,3.0 10.0,6.0,4.0 | cut -d '|' -f 1");
echo $load;
?>
This page will be pulled by AJAX dynamically from static HTML page. You will still have to wait before you will see results however static content will be displayed immediately and dynamic content will follow.
Status page
This page will load AJAX lib and request status for all servers.
CSS style to keep dynamic regions clear:
<style type="text/css">
.clear {
clear: both;
}
</style>
AJAX libs:
<script src="js/ajax.js" type="text/javascript"></script> <script src="js/ajax-dynamic-content.js" type="text/javascript"></script>
Placeholders:
<div id="host1.abc.com "><!-- Empty div for dynamic content --></div> <div id="host2.abc.com "><!-- Empty div for dynamic content --></div>
Last thing is to call AJAX:
<script type="text/javascript">
ajax_loadContent('host1','status.php?host=host1.dns.com');
ajax_loadContent('host2','status.php?host=host2.dns.com');
</script>
Whole page:
!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html>
<head>
<title>Dynamic status</title>
<style type="text/css">
.clear {
clear: both;
}
</style>
<script src="js/ajax.js" type="text/javascript"></script>
<script src="js/ajax-dynamic-content.js" type="text/javascript"></script>
</head>
<body>
<div id="host1"><!-- Empty div for dynamic content --></div>
<div id="host2"><!-- Empty div for dynamic content --></div>
<script type="text/javascript">
ajax_loadContent('host1','status.php?host=host1.dns.com');
ajax_loadContent('host2','status.php?host=host2.dns.com');
</script>
</body>
</html>
Here is how the page works for cluster of 18 servers:
This can expand to more than one monitor per server and other types of tweaks as indications of error etc. Sky’s the limit 
Good luck !
