Posts Tagged ‘nagios’

Using Nagios NIBs with ESX

Thursday, March 22nd, 2012

What is a MIB

A MIB is a Management Information Base. It is an index based upon a network standard that categorizes data for a specific device so that SNMP servers can read the data.

Where to Obtain VMware vSphere MIBs

VMware MIBs are specific to VMware Version, you can try to use the ESX MIBs for ESXi. They can be downloaded from http://downloads.vmware.com. Click on VMware vSphere > find the version of ESX that you are running under “Other versions of VMware vSphere” (the latest version will be the page that you’re on). Click on “Drivers & Tools”. Then click on “VMware vSphere x SNMP MIBs” where “x” is your version.

How to add VMware vSphere MIBs into Nagios

  • Download the VMware vSphere MIBs from http://downloads.vmware.com
  • Copy the MIB files to /usr/share/snmp/mibs/
  • Run check_snmp -m ALL so it detects the new MIBs

Editing snmpd.conf and starting snmpd on ESX

  • Stop snmpd: service snmpd stop
  • Backup snmp.xml: cp /etc/vmware/snmp.xml /etc/vmware/snmp.xml.old
  • Edit snmp.xml with your favorite CLI text editor to have the following:

<config>
  <snmpSettings>
    <communities>public</communities>
    <enable>true</enable>
    <port>171</port>
    <targets>127.0.0.1@162/public</targets>
  </snmpSettings>
</config>

  • Backup snmpd.conf: cp /etc/snmp/snmpd.conf /etc/snmp/snmpd.conf.old
  • Use your favorite CLI text editor and edit /etc/snmp/snmpd.conf
  • Erase everything in it.
  • Add in the following and save it:

load  99 99 99
syslocation ServerRoom
syscontact  “ESX Administrator”
rocommunity  public
view systemview included .1.3.6.1.4.1.6876
proxy -v 1 -c public 127.0.0.1:171 .1.3.6.1.4.1.6876

  • Change “syslocation” and “syscontact” to whatever you want
  • Save your work
  • Configure snmpd to autostart: chkconfig snmpd on
  • Allow SNMP through firewall: esxcfg-firewall –e snmpd
  • Start the SNMP daemon: service snmpd start
  • Restart the mgmt-vmware service: service mgmt-vmware restart

Determining OID

OID’s are MIB specific variables that you can instruct an SNMP server monitor to look for. These variables can be determined by reading the MIBs. One tool that assists with doing this is MIB Browser by iReasoning Networks http://tl1.ireasoning.com/mibbrowser.shtml. MIB Browser can run on Windows, Mac OS X, and Linux/UNIX. To obtain the appropriate OID’s:

  • Load the MIBs in MIB Browser by going to File > Load Mibs
  • Manually comb through to find the OID you want (it will be connected to a string that will be similar to wording used in VSphere).

Example:

  • SNMP MIBs was downloaded from http://downloads.vmware.com for ESX 4.1
  • Loaded MIB for VMWARE-RESOURCES-MIB into MIB Browser
  • Searched for “Mem” (Edit > Find in MIB Tree), found “vmwMemAvail”, the OID for this is .1.3.6.1.4.1.6876.3.2.3.0 (use the OID shown in the dropdown that is near the menu in the MIB Browser – it will show the full OID which will sometimes include a “0″ at the end that the OID listed towards the bottom of the window will not)
  • Add OID into remotehost.cfg (or linux config file) file in Nagios

define service{
use             generic-service ; Inherit values from a template
host_name           ESX4_1
service_description  Memory Available
check_command       check_snmp!-C public -o .1.3.6.1.4.1.6876.3.2.3.0 -m all
}

host_name: the name of the device (whatever you want to call it)
service_description: the name of the service you are monitoring (whatever you want to call it)
check_command: -C is to define the community SNMP string, -o is to define the OID to read, -m is to define which MIB files to load – to be more specific, for this example you can narrow “-m all” to “-m VMWARE-RESOURCES-MIB.MIB”

Once you’ve done the above you should be able to monitor “Memory Available” for ESX through Nagios.  Repeat the procedure, changing steps where applicable for the specific OID you want to monitor.  If you have questions, or need assistance, please contact 318, Inc. at 1-877-318-1318.

Monitoring Xsan with Nagios and SNMP

Monday, December 12th, 2011

Monitoring a system or device using SNMP (a SonicWALL, for instance) is simple enough, provided you have the right MIB. XSNMP is an Open Source project that provides a simple Preference Pane to manage SNMP on OS X, and it also includes an MIB developed by LithiumCorp. This MIB provides OS X’s SNMP agent to gather and categorize information relating specifically to Mac OS X, Mac OS X Server, and Xsan.

XSNMP-MIB can be downloaded from GitHub, or directly from Lithium.

Download the XSNMP-MIB.txt file and put it in /usr/share/snmp/mibs. You can verify that the MIB is loaded by running snmpwalk on the system, specifying the XSNMP Version OID. If snmpwalk returns the version, the MIB is installed correctly. If it returns an error about an “Unknown Object Identifier”, then the MIB isn’t installed in the right spot.

bash$ snmpwalk -c public -v 1 my.server.address XSNMP-MIB::xsnmpVersion
XSNMP-MIB::xsnmpVersion.0 = Gauge32: 1

The fact that the MIB was developed by Lithium doesn’t stop us from using it with Nagios, though. You can define a Nagios service to gather the free space available on your Xsan volume by adding the following to a file called xsan_usage.cfg. Put the file in your Nagios config directory.

define service{
host_name xsan_controller
service_description Xsan Volume Free Space
check_command check_snmp!-C public -o xsanVolumeFreeMBytes.1 -m XSNMP-MIB
}

The host_name should match the Nagios host definition for your Xsan Controller. The service_description can be any arbitrary string that makes sense and describes the service.

The check_command definition is the actual command that’s run. The -C flag defines the SNMP community string, the -m flag defines which MIB should be loaded (you can use “-m all” to just load them all), and the -o flag defines which OID we should return. “xsanVolumeFreeMBytes.1″ should return the free space, in MB, of the first Xsan volume.

Using Nagios MIBs with a SonicWALL

Wednesday, November 23rd, 2011

MIB (short for Management Information Base), is an index based on a network standard that categorizes data for a specific device so SNMP servers can read the data. SonicWALL MIBs are specific to device AND firmware.  Each can be downloaded from www.mysonicwall.com (you will need to have an account to download).  Click on Downloads, Download Center and then find the firmware that you are running.  Then click on “SNMP MIBs” to download.

Once downloaded, copy the MIB files to /usr/share/snmp/mibs to prepare them for loading into NetSNMP. Then run check_snmp with a -m option followed by ALL so that Nagios will detect the new MIBs:
check_snmp -m ALL
Once complete, determine the OID. OID’s are MIB variables that instruct an SNMP server monitor to look for information on the device. These variables can be determined by reading the MIBs.  One tool that assists with doing this is MIB Browser by iReasoning Networks http://tl1.ireasoning.com/mibbrowser.shtml  MIB Browser can run on Windows, Mac OS X, and Linux/UNIX.  To obtain the appropriate OID’s:
  1. Load the MIBs in MIB Browser by going to File > Load Mibs
  2. Manually comb through to find the OID you want (a string used in the SonicWALL Web Configuration).

To put this into use, let’s prepare an snmpwalk from a TZ100. First, download the SNMP MIBs from MySonicWALL.com for a TZ100 running firmware version (5.6.0.12-65o). Then let’s load the MIB for SONICWALL-FIREWALL-IP-STATISTICS-MIB into MIB Browser. Searching for “CPU” (Edit -> Find in MIB Tree) shows sonicCurrentCPUUtil, the OID for this fact is .1.3.6.1.4.1.8741.1.3.1.3.0. We used the OID shown in the drop-down near the menu in the MIB Browser. This shows the full OID, which sometimes includes a “0″ at the end (shown towards the bottom of the window). Next, add the OID into a switch.cfg file in nagios:

define service{
use                                       generic-service ; Inherit values from a template
host_name                       TZ100
service_description     CPU Utilization
check_command           check_snmp!-C public -o .1.3.6.1.4.1.8741.1.3.1.3.0 -m all
}

These settings include the following:

  • host_name: the name of the device (whatever you want to call it)
  • service_description: the name of the service you are monitoring (whatever you want to call it)
  • check_command: -C is to define the community SNMP string, -o is to define the OID to read, -m is to define which MIB files to load – to be more specific, for this example you can narrow “-m all” to “-m SONICWALL-FIREWALL-IP-STATISTICS-MIB.MIB”

Overall, setting up Nagios to be able to leverage MIBs from 3rd party vendors is an easy task, if not tedious when there are a lot of settings you’d like to walk through with SNMP.

Installing Lithium on Mac OS X

Thursday, November 1st, 2007

Installing Lithium Core 4.9.0 Make sure the system is not currently a web server and port 80 is available. Download the Lithium 4.9.0 package. Double-click on the Core 4.9.0 Installer. Click Continue through the license agreement screens. Choose the packages to install and click on Continue. Choose the location to install the Lithium Core application and click on Install. Enter the credentials of an administrator and Click OK. When the installer is complete, click on the Close button. Open Lithium Core Admin from the /Applications folder. Click Next and enter the name of the client for whom you are installing Lithium. Click Next and enter a new administrative username and password for accessing Lithium. Click Next and you will be placed into the database configuration screen. Unless you are using PostgreSQL on another host, do not modify these settings. Click Next and double-check the settings. If they look good then click on the Finish button and enter administrative credentials to commit the changes. When you open Lithium Console from the /Applications folder for the first time you will be asked whether you would like to check for updates each time. Click Yes. You have now installed Lithium and can move on to adding hosts to be monitored.

Using NPRE with Nagios

Wednesday, July 5th, 2006

Nagios is a computer Monitoring Software. Nagios runs on one central server, and has the ability to check resources on remote computers. In order to allow these remote computers to be monitored the Nagios Software team has NRPE. NRPE functions as a Daemon and plugin for executing plugins on remote hosts. When installed on the remote server it creates a Medium for the Nagios server to be able to execute commands to the remote agent. You have the ability to check a wide range of resources, CPU, Hard Drive Space, Computer Load, Server Services, DNS checks etc.

To Download NRPE and NRPE Plugins

http://www.nagios.org/download/

In this example we will be installing NRPE to a remote Linux Server. So to send the downloaded file to the remote server
scp Documents/NAGIOS/nagios-plugins-1.4.9.tar.gz USER@HOSTNAME:
scp Documents/NAGIOS/nrpe-2.8.1.tar.gz USER@HOSTNAME:
This command will send the downloaded file to the 318admin accounts home folder on the remote linux server.

NRPE Plugins
Once the file has been copied to the server and placed in an appropriate location, for example /usr/local/src
First we will uncompress the package
[root@HOSTNAME src]# tar -xzvf nagios-plugins-1.4.9.tar.gz
Then cd into nagios-plugins-1.4.9
Perform the following to compile and install the plugins

[root@HOSTNAME nagios-plugins-1.4.9]# ./configure
With this executed sucsesfuly you should expect an output similar to this:
config.status: creating po/Makefile
–with-apt-get-command:
–with-ping6-command:
–with-ping-command: /bin/ping -n -U -w %d -c %d %s
–with-ipv6: yes
–with-mysql: /usr/bin/mysql_config
–with-openssl: yes
–with-gnutls: no
–with-perl: /usr/bin/perl
–with-cgiurl: /nagios/cgi-bin
–with-trusted-path: /bin:/sbin:/usr/bin:/usr/sbin

Now we will make the package
[root@HOSTNAME nagios-plugins-1.4.9]# make
With this executed successfully the following
Making all in po

Now we will perform the make install command to install the Nagios Plugins on the remote server. Run this command as root
[root@HOSTNAME nagios-plugins-1.4.9]# make install

Nagios Documentation recommends changing the permissions on the files after the install is performed. The commands to perform this are as follows:
chown nagios:nagios /usr/local/nagios
chown -R nagios:nagios /usr/local/nagios/libexec

Now we will need to Install the NRPE Daemon to execute these plugins

NRPE Daemon
Next we will uncompress the package on the remote server. This document assumes that you have some type of root access to the server.
First we will uncompress the file, I generally move my source files to /usr/local/src
To complete these tasks first SSH into the remote server
Sudo mv nrpe-2.8.1.tar.gz /usr/local/src
Cd /usr/local/src
sudo tar -xzvf nrpe-2.8.1.tar.gz

We have now moved the package file to /usr/local/src and uncompressed the package to /usr/local/src/nrpe-2.8.1

Next a nagios user account needs to be added on the remote system. Execute this command as sudo or root
Useradd nagios
Passwd nagios

Next we must configure and compile the package before we can install it. I have found that compiling NRPE to work best when executed as root, perhaps this will be resolved in later versions. For the purspose of this documentation, it is assumed that the following commands are executed as root

Cd /usr/local/src/nrpe-2.8.1
[root@HOSTNAME nrpe-2.8.1]# ./configure

When ./configure executes properly you should see an output similar to this
*** Configuration summary for nrpe 2.8.1 05-10-2007 ***:

General Options:
————————-
NRPE port: 5666
NRPE user: nagios
NRPE group: nagios
Nagios user: nagios
Nagios group: nagios

Review the options above for accuracy. If they look okay,
type ‘make all’ to compile the NRPE daemon and client.

If you receive errors do not proceed, you must resolve what ever dependency errors you are receiving when attempting to compile.

Next we will perform the make command to ‘make’ the installation. This is done by the following command
[root@HOSTNAME nrpe-2.8.1]# make all

With a succesfull make an output similar to this should be seen
*** Compile finished ***

If the NRPE daemon and client compiled without any errors, you
can continue with the installation or upgrade process.

We are now ready to install NRPE on the remote system. To perform the install, execute the following commands as root:
[root@phillip2 nrpe-2.8.1]# make install-plugin
[root@phillip2 nrpe-2.8.1]# make install-daemon
[root@phillip2 nrpe-2.8.1]# make install-daemon-config
[root@phillip2 nrpe-2.8.1]# make install-xinetd

Now since we installed NRPE with Xinetd, we have to edit the following nrpe file to allow outside connections to NRPE from the Nagios Server
Perform this as root
[root@HOSTNAME nrpe-2.8.1]# nano /etc/xinetd.d/nrpe
Now under ‘only_from’ add the local IP address of the Nagios Server, for example you can have the following:
only_from = 127.0.0.1 192.168.1.23
Save the changes and exit the document edit program

Now we have to Edit the Services file to add the port for NRPE to run. Perform the Following as root:
[root@HOSTNAME nrpe-2.8.1]# nano /etc/services
At the bottom of the File I would just add the following Text
nrpe 5666/tcp #NRPE

Once we have edited the Xinetd File and Services file, perform a restart of Xinetd to apply the changes
This should look something like this if done correctly:
[root@HOSTNAME nrpe-2.8.1]# service xinetd restart
Stopping xinetd: [ OK ]
Starting xinetd: [ OK ]
To test that the NRPE Daemon is running you can perform this command
netstat -at|grep nrpe
tcp 0 0 *:nrpe *:* LISTEN

If you do not see the same output Nagios Documentation recommends to check the following:
– You added the nrpe entry to your /etc/services file
– The only_from directive in the /etc/xinetd.d/nrpe file contains an entry for “127.0.0.1″
– xinetd is installed and started
– Check the system log files for references about xinetd or nrpe and fix any problems that are reported

Also you can use the check_nrpe plugin to check your installation of NRPE by performing the following command:
[root@HOSTNAME nrpe-2.8.1]# /usr/local/nagios/libexec/check_nrpe -H localhost
NRPE v2.8.1

The service and Version number should be the expected output of this command.

Before we go to the Nagios server to execute the plugins, in most cases we just have to confirm that the correct devices are being called by the plugins in the config file. Most common is the incorrect hard drive will be checked, which makes the tool not as useful.

So for this server when I run the df command I would want Nagios to examine the partitions of / and /home

For this server I want NRPE to monior /dev/sda1 and /dev/sdb1 so to make this happen we will edit the nrpe.cfg file, /usr/local/nagios/etc/nrpe.cfg down at the bottom are the check commands connected to devices. I have added these two lines, and removed any other hard drive devices
command[check_sda2]=/usr/local/nagios/libexec/check_disk -w 20 -c 10 -p /dev/sda2
command[check_sdb1]=/usr/local/nagios/libexec/check_disk -w 20 -c 10 -p /dev/sdb1

-w 20 means to warn when the drive is 20% free space available
-c 10 mean to have a critical warning when 10% free space available

Configure Nagios Server to Monitor Remote Host

The Remote server must already be running Nagios, this documentation assumes this has already been done. Also if you will need to install the chek_nrpe plugin on the Nagios server. The is done in the same way that a remote host would have NRPE installed, which has already been covered.

After all the requirements have been met, its now time to define server commands for the new host.
First you have to define a services file. This is done on the Nagios Server nagios.cfg. Edit the file located in /usr/local/nagios/etc/nagios.cfg and include for example
cfg_file=/usr/local/nagios/etc/hosts.cfg
cfg_file=/usr/local/nagios/etc/services.cfg
cfg_file=/usr/local/nagios/etc/timeperiods.cfg

Then we will edit the Host.cfg to include the new remote host. Using a text editor add the following, this will add a Host called HOSTNAME with an a local ip of 192.168.1.226, also it will be given the linux-server template. Templates are configured in the nagios.cfg file

define host{
use linux-server
host_name HOSTNAME
alias HOSTNAME
address 192.168.1.226
}

Now we can edit the services file to tell Nagios what NRPE services to check on the new Remote Host. I generally use this template to check various default services

#HOSTNAME
define service{
use generic-service
host_name HOSTNAME
service_description / Free Space
is_volatile 0
check_period 24×7
max_check_attempts 3
normal_check_interval 3
retry_check_interval 1
contact_groups admins
notification_interval 120
notification_period 24×7
notification_options w,u,c,r
check_command check_nrpe!check_sda2
}

define service{
use generic-service
host_name HOSTNAME
service_description /home Free Space
is_volatile 0
check_period 24×7
max_check_attempts 3
normal_check_interval 3
retry_check_interval 1
contact_groups admins
notification_interval 120
notification_period 24×7
notification_options w,u,c,r
check_command check_nrpe!check_sdb1
}

define service{
use generic-service ; Name of service template to use
host_name HOSTNAME
service_description SMTP
is_volatile 0
check_period 24×7
max_check_attempts 3
normal_check_interval 3
retry_check_interval 1
contact_groups admins
notification_interval 120
notification_period 24×7
notification_options w,u,c,r
check_command check_smtp
}
define service{
use generic-service ; Name of service template to use
host_name HOSTNAME
service_description FTP
is_volatile 0
check_period 24×7
max_check_attempts 3
normal_check_interval 3
retry_check_interval 1
contact_groups admins
notification_interval 120
notification_period 24×7
notification_options w,u,c,r
check_command check_ftp
}
define service{
use generic-service ; Name of service template to use
host_name HOSTNAME
service_description HTTP
is_volatile 0
check_period 24×7
max_check_attempts 3
normal_check_interval 3
retry_check_interval 1
contact_groups admins
notification_interval 120
notification_period 24×7
notification_options w,u,c,r
check_command check_nrpe!check_http
}
define service {
use generic-service
host_name HOSTNAME
service_description CPU Load
is_volatile 0
check_period 24×7
max_check_attempts 3
normal_check_interval 3
retry_check_interval 1
contact_groups admins
notification_interval 120
notification_period 24×7
notification_options w,u,c,r
check_command check_nrpe!check_load
}
define service{
use generic-service
host_name HOSTNAME
service_description Current Users
is_volatile 0
check_period 24×7
max_check_attempts 3
normal_check_interval 3
retry_check_interval 1
contact_groups admins
notification_interval 120
notification_period 24×7
notification_options w,u,c,r
check_command check_nrpe!check_users
}
define service{
use generic-service
host_name HOSTNAME
service_description Total Processes
is_volatile 0
check_period 24×7
max_check_attempts 3
normal_check_interval 3
retry_check_interval 1
contact_groups admins
notification_interval 120
notification_period 24×7
notification_options w,u,c,r
check_command check_nrpe!check_total_procs
}
define service{
use generic-service
host_name HOSTNAME
service_description Zombie Processes
is_volatile 0
check_period 24×7
max_check_attempts 3
normal_check_interval 3
retry_check_interval 1
contact_groups admins
notification_interval 120
notification_period 24×7
notification_options w,u,c,r
check_command check_nrpe!check_zombie_procs
}

After you have edited the host and service files, you may want to run a check to see what if anything is wrong with your config. Run this file, I find it helpful to make this just a script that you can execute once your done. The command is
/usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg

If everything runs without error, its safe to restart the nagio service and check the website for the new host.

How to Monitor Remote services with Nagios and NRPE

As long as both the Nagios server and Remote Host running NRPE have the nagios plugins installed this is very straight forward.

In this example we will add monitoring of HTTP and MySQL to a remote server named terrence2.

First on the Remote Host with NRPE, edit the nrpe.cfg file to include:
command[check_mysql]=/usr/local/nagios/libexec/check_mysql -h localhost
command[check_http]=/usr/local/nagios/libexec/check_http -h localhost

Then on the Nagios server simply edit the services.cfg file to add the following check command:
define service{
use generic-service ; Name of service template to use
host_name HOSTNAME
service_description HTTP
is_volatile 0
check_period 24×7
max_check_attempts 3
normal_check_interval 3
retry_check_interval 1
contact_groups admins
notification_interval 120
notification_period 24×7
notification_options w,u,c,r
check_command check_nrpe!check_http

and for MySQL

define service{
use generic-service ; Name of service template to use
host_name HOSTNAME
service_description MySQL
is_volatile 0
check_period 24×7
max_check_attempts 3
normal_check_interval 3
retry_check_interval 1
contact_groups admins
notification_interval 120
notification_period 24×7
notification_options w,u,c,r
check_command check_nrpe!check_mysql

Restart xinetd on the remote host and restart nagios on the server and your up and monitoring.