Friday, February 12, 2010

Sending SNMP Traps from Nagios

As well as receiving SNMP traps in Nagios, you can send SNMP traps from Nagios to a remote SNMP management station like HP OpenView (NNM) or the like. The easiest way to do this is to create a notification command that generates an SNMP trap. This way, you can use an SNMP management station as a notification destination. This will mean that the trap will be sent every time a notification is scheduled.


Prerequisites:

net-snmp, net-snmp-utils should be installed on Monitoring host running Nagios

Connectivity between management station and nagios has been stablished and nagios can connect and send snmptraps to snmptrap 162/udp port on management station.


1. First of all define notification command, below is the commands for service and hosts in commands.cfg

# 'send-service-trap' command definition
define command{
      command_name send-service-trap
      command_line /usr/local/bin/send-service-trap manager public "$HOSTNAME$" "$SERVICEDESC$" $SERVICESTATEID$ "$SERVICEOUTPUT$"
}

# 'send-host-trap' command definition
define command{
                command_name send-host-trap
                command_line /usr/local/bin/send-host-trap manager public "$HOSTNAME$" $HOSTSTATEID$ "$HOSTOUTPUT$"
}

Both the commands call the shell scripts send-service-trap and send-host-trap in /usr/local/bin/. Passing the hostname or IP address of the destination management station and the target community string, manager and public, respectively.

A number of macros are also passed to the shell script representing the hostname, the service description, the service/host state in numeric form, and the output of the host/service check.

The snmptrap tool generates SNMP traps and can send them to remote management stations. The traps are generated according to a Nagios
MIB available from the same Sourceforge project as the Nagios plug-ins. You can download the MIB from http://prdownloads.sourceforge.net/nagiosplug/nagiosmib-1.0.0.tar.gz?download.

The package contains two MIB files: a root MIB file containing basic definitions for Nagios OIDs called NAGIOS-ROOT-MIB and an MIB containing events and traps called NAGIOS-NOTIFY-MIB. You will need to install both MIB files into your SNMP management device; for example, if your trap daemon is the snmptrapd daemon, you would generally copy these files to the /usr/share/snmp/mibs directory. When you start the snmptrapd daemon, you load the MIB files using the -m
and -M options:

Also copy the MIBs files on localhost where Nagios is running under snmp/mibs

/usr/sbin/snmptrapd -m ALL -M /usr/share/snmp/mibs -Lf /var/log/snmptrapd.log


Tip:  There is an excellent HOWTO on how to use snmptrap to send traps here.

Note: Your own SNMP management station, such as HP OpenView or the like, will have its own method of loading MIB files. You should refer to its documentation for that.

2. Define a contact for Management station

Need to define a contact for management station in the same way we create for other contact person, to receive SNMP traps from Nagios whereever Nagios send notificaiton for host/service

define contact{
        contact_name                             managementstation
        use                                              generic-contact
        alias                                            Management Station
        service_notification_options    w,u,c,r
        host_notification_options         d,u,r
        service_notification_commands   send-service-trap
        host_notification_commands      send-host-trap
        }


3. Add the new conatct to contactgroup for hosts/services for which you want to send SNMP traps

define contactgroup{
        contactgroup_name       linuxadmins
        alias                                Linux Servers Administrators
        members                         me, managementstation
}

Now whenever there is a alert for linux servers Nagios will notify contact "managementstation" which in turn call the commands send-service-trap/send-host-trap accordingly depends on host or service alert.

4. send-service-trap/send-host-trap Shell Script

===/usr/local/bin/send-service-trap ====
# Arguments:
# $1 = Management Station
# $2 = Community String
# $3 = host_name
# $4 = service_description (Description of the service)
# $5 = return_code (An integer that determines the state
#       of the service check, 0=OK, 1=WARNING, 2=CRITICAL,
#       3=UNKNOWN).
# $6 = plugin_output (A text string that should be used
#       as the plugin output for the service check)
#
#
/usr/bin/snmptrap -v 2c -c $2 $1 '' NAGIOS-NOTIFY-MIB::nSvcEvent nSvcHostname s "$3" nSvcDesc s "$4" nSvcStateID i $5 nSvcOutput s "$6"


===/usr/local/bin/send-host-trap=======
# Arguments:
# $1 = Management Station
# $2 = Community String
# $3 = host_name
# $4 = HostStatID A number that corresponds to the current state of the host: 0=UP, 1=DOWN, 2=UNREACHABLE.
# $5 = HOSTOUTPUT The first line of text output from the last host check (i.e. "Ping OK").
#
#
/usr/bin/snmptrap -v 2c -c $2 $1 '' NAGIOS-NOTIFY-MIB::nHostEvent nHostname s "$3" nHostStateID i $4 nHostOutput s "$5"


The shell script receives all the incoming variables and passes them to the snmptrap command

-v = snmp version
-c = community string
$1 = Management Station IP address or hostname

The next two single quotes are special characters representing the uptime portion of a trap. They will be replaced with the current uptime of the system generating the trap when the snmptrap command is executed.

The enterprise OID is specified next; in this case it is NAGIOS-NOTIFY-MIB::nSvcEvent and NAGIOS-NOTIFY-MIB::nHostEvent. The NAGIOS-NOTIFY-MIB is the name of the MIB module being referenced, and the nSvcEvent/nHostEvent is the trap defined for sending service events. Together these form the enterprise OID.

Next specified a list of individual OIDs and their variables that I’m passing as part of the trap. They are, in order, the hostname, the service description, the host/service status ID in numerical form, and the output of the host/service check.

Enclosing the values ones that might contain multiword data in quotation marks so that they are passed cleanly to the command.

Tip:  You can find these and other OIDs that you can use with the traps and notifications defined in the NAGIOS-ROOT-MIB and NAGIOS-NOTIFY-MIB MIB files.

23 comments:

  1. I have successfully configured sending SNMP traps from Nagios to HP NNM.

    ReplyDelete
  2. Thank you for your very helpful posting. It enabled me to get Nagios sending traps very quickly.

    ReplyDelete
  3. I Configured everthing you mentioned in your perfect tutorial but i even cannot send snmptraps via the snmptrap command snmptrap -v 1 -c public manager "" "" 6 99 ""

    ReplyDelete
  4. Everything is fine now. I was not able to receive snmptraps from nagios on the other management station because the snmpd version 5.4.rc2 on nagios had a bug. The bug was that snmptrap sends its traps on port 161 and not on 162.

    ReplyDelete
  5. N.B. typo:
    smnptrapd.log should be snmptrapd.log!

    ReplyDelete
  6. Hi Askar,
    Thanks for the very straight forward steps to be followed. 1 thing is missing, I'm a guy new to nagios maybe because of that..I configured nagios and it monitors my network and send email alerts. Now I need to send alerts in form of traps to 'Management Host' in my parent company. So I have just done all that you presented here. Am I done or should I make some more changes?
    Should I replace the arguments on the script below OR where do I mention the management host IP address.
    Can I check the traps in my own nagios machine as well??
    ===/usr/local/bin/send-host-trap=======
    # Arguments:
    # $1 = Management Station
    # $2 = Community String
    # $3 = host_name
    # $4 = HostStatID A number that corresponds to the current state of the host: 0=UP, 1=DOWN, 2=UNREACHABLE.
    # $5 = HOSTOUTPUT The first line of text output from the last host check (i.e. "Ping OK").
    #
    #
    /usr/bin/snmptrap -v 2c -c $2 $1 '' NAGIOS-NOTIFY-MIB::nHostEvent nHostname s "$3" nHostStateID i $4 nHostOutput s "$5"

    Thanks in Advance!!!
    Hamid

    ReplyDelete
    Replies
    1. Refer to 1) First of all define notification command, below is the commands for service and hosts in commands.cfg

      command_line /usr/local/bin/send-service-trap manager public <-- manager is the hostname of management station, edit /etc/hosts file on your Nagios server and put something like this in it

      # SNMP Manager Station
      xxx.xx.xx.xx manager

      replace xx.xxx. with the IP of your management station, after that nagios should able to resolve 'manager' you can also confirm by..

      ping manager

      If you followed everything correctly Nagios should be forwarding SNMP traps to the management station.

      you need to coordinate with the guy managing the Management Station in your parent company make sure they copied the Nagios MIBs to correct location and loaded.

      Sending and receiving traps in nagios is bit complicated specially to those who are new to Nagios. If stuck i would recommend to Google.

      Delete
  7. I am not much involve with Nagios or open source in general due to nature of my current job and environment. Therefore I wouldn't of great help if someone come up with a question.

    ReplyDelete
  8. Hello Askar,

    This is Mohammed Azhar Hussain I am running Nagios 3.3.1 on centos it working fine but I have one doubt about SNMP-2 when my monitoring switch port down it only shows SNMP-2 but there is no any notification alert. Could you please help me to assign notification on this service.

    Thanks & Regards
    Mohammed Azhar Hussain

    ReplyDelete
  9. Hi Askar,

    how is it possible to change integer type of nHostStateID and nSvcStateID to string type ?
    I'm using SNMPc software which have a problem to recognize state by number but should be work with string.

    Thanks,
    Jiri

    ReplyDelete
    Replies
    1. Hi Jiri,
      I am afraid can't be of much help here due to job switch I am not working on this from last two years.

      Delete
  10. Hi Askar,
    I configured by using your reference, but enterprises OID cannot send. Can you give me a solution.

    Thanks
    Ramana

    ReplyDelete
  11. The MIBS are not available in the path http://prdownloads.sourceforge.net/nagiosplug/nagiosmib-1.0.0.tar.gz?download as per the article. Can they be downloaded from https://www.nagios.org/downloads/nagios-plugins/ ?

    ReplyDelete
    Replies
    1. https://github.com/nagios-plugins/nagios-mib/tree/master/MIB

      Delete
  12. Hello Everyone,

    Sending traps from Nagios side is working fine after following up the steps above. But on the other side (NMS: HP SiteScope) is recognizing the trap as string and didn't recognize the OIDs correctly!
    Can anyone please advise?

    Thanks in advance,,,
    Tamer Moalla

    ReplyDelete
  13. I have deployed Nagios core on one VM. This receives all the traps from my application and forward it to other NMS system. I want to filter some traps not to be forward to NMS system.

    How I can achieve this scenario.

    ReplyDelete
  14. Somewhat helpful, I am still confused as our management wants host availability check using SNMP ? Is it possible ? What will be the result in case of remote host unreachable.

    ReplyDelete
  15. Thanks for this post, realy helpfully for me, i have a question, how i can add values to the trap, i.e. i need sen the host ip address and host father, how i can do it?

    ReplyDelete
  16. This comment has been removed by the author.

    ReplyDelete
  17. Hi,

    I would like to know if is it possible to send SNMP traps from Nagios to another host, using the free version of Nagios (Nagios Core).

    If you look at the next link (https://assets.nagios.com/handouts/nagiosxi/Nagios-XI-vs-Nagios-Core-Feature-Comparison.pdf), it says "Send & Receive SNMP Traps" is a feature of Nagios XI (payment version), not Nagios Core (free version). But if you look at this link (https://www.nagios.com/solutions/snmp-traps/), it says both solutions (Nagios XI and Nagios Core) provide that feature.

    What solution would I need to send SNMP traps to another host? Could I do it using the free version of Nagios? Thanks.

    Cordially,

    Albert

    ReplyDelete
  18. Hi

    I need send generic macros trought snmp trap , Do you konw how can do it ?

    Regards

    ReplyDelete