Getting Data In

Are there any plans to resurrect the Nagios check_splunk plugin?

Explorer

There used to be a Splunk2Nagios application that came with Splunk, and it worked very well. When 4.x was released it was still possible to get most of the functionality of this plugin working with a few changes in the "check_splunk" script, except for checking license usage. Now with 4.1.x this plugin no longer works at all.

Any chance Splunk and Nagios will get together and work on a new version?

Tags (2)
1 Solution

Contributor

I have written a simple shell script that can be triggered from a "Scheduled Saved Search" in Splunk to send alerts to Nagios:

#!/bin/bash
#
# Author: Luke Harris <luke.harris@ce.com.au>
#
# Splunk to Nagios is Awesome.
# This script can be triggered from a Scheduled Saved Search in Splunk to send alerts to Nagios.
# 
# Pre-requisites:
# send_nsca must be installed on your *nix Splunk server
# nsca must be listening on your Nagios server 
# The Saved Search must begin with the corresponding hostname defined in Nagios followed by a hyphen then the Service defined in Nagios, eg. server01 - XYZ Alert
# Time range:
#  Start time (optional)  = -5m@m
#  Finish time (optional) = now
# Schedule and alert:
#  tick "Schedule this search"
# Schedule type = Basic
#  Run every = 5 minutes
# Alert conditions:
#  Perform actions = if number of events
#  is greater than 0
# Alert actions:
#  tick Trigger shell script
# Filename of shell script to execute = splunk-nagios.sh
# Note: must be located in $SPLUNK_HOME/bin/scripts
#
# version: 2010052001
#
SPLUNKSERVER=splunk01
WWW=splunk
NSCABIN=/usr/lib/nagios/plugins
NSCACFG=$NSCABIN
NSCAHOST=nagios.abc.com.au
NSCAPORT=5667
NSCA_SERVICE_NAME=`echo $5|awk -F'[' '{print $2}'|awk -F']' '{print $1}'|sed 's/\- //g'|cut -f2- -d " "`
HOST=`echo $5|awk -F'[' '{print $2}'|awk -F']' '{print $1}'|awk '{print $1}'|tr A-Z a-z`
EVENTS=$1
URL=`echo $6|sed "s/$SPLUNKSERVER/$WWW/"`
ERROR=`zcat $8|sed -n '2p'`
NSCA_MSG="$5 Error: $ERROR URL: $URL"


nagios_notify () {
${NSCABIN}/send_nsca -H $NSCAHOST -p $NSCAPORT -d "," -c $NSCACFG/send_nsca.cfg <<EOF
$HOST,$NSCA_SERVICE_NAME,$NSCA_CODE,$NSCA_MSG
EOF
}

if [ $EVENTS -gt 0 ]
 then
        NSCA_CODE=1
        nagios_notify
 else
NSCA_CODE=0
exit 0
fi

I used the "Configure scripted alerts" reference in the Admin Manual:
http://www.splunk.com/base/Documentation/latest/Admin/Configurescriptedalerts

I welcome any feedback and am happy to answer any further questions or queries.

FYI: we also have a cron job that rsync's the nagios.log from our Nagios server to our Splunk server for ingestion.

Luke 🙂

View solution in original post

Splunk Employee
Splunk Employee

Luke - thanks for writing that up. Would you be interested in posting it to Splunkbase?

Thanks! John Mark Splunk Community Guy

0 Karma

Contributor

I have written a simple shell script that can be triggered from a "Scheduled Saved Search" in Splunk to send alerts to Nagios:

#!/bin/bash
#
# Author: Luke Harris <luke.harris@ce.com.au>
#
# Splunk to Nagios is Awesome.
# This script can be triggered from a Scheduled Saved Search in Splunk to send alerts to Nagios.
# 
# Pre-requisites:
# send_nsca must be installed on your *nix Splunk server
# nsca must be listening on your Nagios server 
# The Saved Search must begin with the corresponding hostname defined in Nagios followed by a hyphen then the Service defined in Nagios, eg. server01 - XYZ Alert
# Time range:
#  Start time (optional)  = -5m@m
#  Finish time (optional) = now
# Schedule and alert:
#  tick "Schedule this search"
# Schedule type = Basic
#  Run every = 5 minutes
# Alert conditions:
#  Perform actions = if number of events
#  is greater than 0
# Alert actions:
#  tick Trigger shell script
# Filename of shell script to execute = splunk-nagios.sh
# Note: must be located in $SPLUNK_HOME/bin/scripts
#
# version: 2010052001
#
SPLUNKSERVER=splunk01
WWW=splunk
NSCABIN=/usr/lib/nagios/plugins
NSCACFG=$NSCABIN
NSCAHOST=nagios.abc.com.au
NSCAPORT=5667
NSCA_SERVICE_NAME=`echo $5|awk -F'[' '{print $2}'|awk -F']' '{print $1}'|sed 's/\- //g'|cut -f2- -d " "`
HOST=`echo $5|awk -F'[' '{print $2}'|awk -F']' '{print $1}'|awk '{print $1}'|tr A-Z a-z`
EVENTS=$1
URL=`echo $6|sed "s/$SPLUNKSERVER/$WWW/"`
ERROR=`zcat $8|sed -n '2p'`
NSCA_MSG="$5 Error: $ERROR URL: $URL"


nagios_notify () {
${NSCABIN}/send_nsca -H $NSCAHOST -p $NSCAPORT -d "," -c $NSCACFG/send_nsca.cfg <<EOF
$HOST,$NSCA_SERVICE_NAME,$NSCA_CODE,$NSCA_MSG
EOF
}

if [ $EVENTS -gt 0 ]
 then
        NSCA_CODE=1
        nagios_notify
 else
NSCA_CODE=0
exit 0
fi

I used the "Configure scripted alerts" reference in the Admin Manual:
http://www.splunk.com/base/Documentation/latest/Admin/Configurescriptedalerts

I welcome any feedback and am happy to answer any further questions or queries.

FYI: we also have a cron job that rsync's the nagios.log from our Nagios server to our Splunk server for ingestion.

Luke 🙂

View solution in original post

Contributor

I have uploaded a new app to Splunkbase called SplunkForNagios - it has been developed to present events from the Open Source monitoring solution "Nagios" in Splunk, giving you the added ability to correlate problems in your environment and even trigger alerts from Splunk to Nagios. Over 40 field extractions are included with SplunkForNagios, as well as 6 saved searches, and an advanced dashboard featuring recent Warning and Critical Alerts.

Download link: http://www.splunkbase.com/apps/All/4.x/Add-On/app:SplunkForNagios

0 Karma

Contributor

We definitely have a lot of customers asking for Nagios integration. The things we hear most are:

  • raising Nagios alerts in response to Splunk alerts.
  • logging alerts raised by Nagios into Splunk, to correlate problems reported by nagios with root-cause log data available in Splunk.
  • logging raw data collected by Nagios into Splunk, to enable Splunk to report on historical trends collected by nagios and to correlate monitoring data collected by nagios with log data collected by Splunk.

Does that list sounds like the right set of things to focus on? (you can tack a comment onto this answer with your response)