All Apps and Add-ons

How to create event in Zenoss v.5 which listens only on HTTPS?

New Member

Hello,

Zenoss v.5 listens only on HTTPS for any incoming event.
'Create_event' function of this add-on doesn't seem to handle SSL, so HTTPS connection fails.
Any update/work-around for this?

thanks,

0 Karma

Splunk Employee
Splunk Employee

The current version of the app v2.0.2 is compatible with Zenoss 5.x

https://splunkbase.splunk.com/app/2766/#/details

0 Karma

Motivator

I've created a shell script for Splunk to run as an alert trigger action which will loop through events and post to the Zenoss api.
Note: you may have to customize for fields relevant to your splunk and zenoss environments respectively.

If this script is not working, verify that you have placed it in $SPLUNK_HOME/bin/scripts and have restarted splunk
Alerts should reference zenoss-alert-forward-wrapper.sh which enables logging and calls zenoss-alert-forward.sh

Template Search:

index=main
| head 1 
| eval severity="INFO" 
| eval resource=host_fqdn 
| eval component=sourcetype+":"+source 
| eval evclass="/App/Splunk" 
| eval summary="Splunk to Zenoss test pulse" 
| eval comment="--------------------------------------------------------------------------------" 
| eval comment="valid values for severity: CRITICAL ERROR WARNING INFO DEBUG CLEAR" 
| eval comment="resource must be host_fqdn if wanting to link to a host object in Zenoss or we may have to have a Zenoss script to do the conversion?"
| eval comment="component is currently set to soucetype + source, however can be customized. Check with your Zenoss admin"
| eval comment="evclass should always be /App/Splunk unless the Zenoss admin asks you to change it" 
| eval comment="table command is order-sensitive i.e. last command in an alert sent to zenoss should always be | table severity resource component evclass summary" 
| eval comment="--------------------------------------------------------------------------------" 
| table severity resource component evclass summary
| fillnull value=missing_value

zenoss-alert-forward-wrapper.sh

#!/usr/bin/env sh
# export all vars so the child script will have access to them
# if this script is not working, verify that you have placed it in $SPLUNK_HOME/bin/scripts and have restarted splunk

export SEARCH_COUNT="$1"       # $1 - Number of events returned
export SEARCH_TERMS="$2"       # $2 - Search terms
export SEARCH_QUERY="$3"       # $3 - Fully qualified query string
export SEARCH_NAME="$4"        # $4 - Name of saved search
export SEARCH_REASON="$5"      # $5 - Reason saved search triggered
export SEARCH_URL="$6"         # $6 - URL/Permalink of saved search
export SEARCH_TAGS="$7"        # $7 - Always empty as of 4.1
export SEARCH_PATH="$8"        # $8 - Path to raw saved results in Splunk instance (advanced)

export RESULTS_GZ="$SEARCH_PATH"

SCRIPT_DIR=$SPLUNK_HOME/bin/scripts
LOG_FILE=$SPLUNK_HOME/var/log/splunk/zenoss-alert-forward.log

# execute script and log standard out and standard error 
$SCRIPT_DIR/zenoss-alert-forward.sh 2>&1 > $LOG_FILE

zenoss-alert-forward.sh

  #!/usr/bin/env sh
# zenoss-alert-forward.sh
# read through splunk results and generate an alert per splunk event to push to zenoss api via curl
# if this script is not working, verify that you have placed it in $SPLUNK_HOME/bin/scripts and have restarted splunk

# rob@jordan2000.com
# version 1.0.5
# created 8/23/2017
# updated 11/7/2018

# Parameters passed in from the alert.
# $1-$9 is the positional parameter list.

# --------------------------------------------------------------------------------
# *** CONFIG PARAMS
# --------------------------------------------------------------------------------

 ZENOSS_API_USER="your_zenoss_api_user_here"
 ZENOSS_API_PASS="your_zenoss_api_pass_here"
 ZENOSS_URL="https://your.zenoss.api.url.here:443"
ZENOSS_API_PATH="/zport/dmd/evconsole_router"
# maximum number of events that can be sent to Zenoss per Splunk alert
# each Splunk event will be sent as individual alert
# anything over the event limit will be truncated
EVENT_LIMIT=100

# in most cases, you shouldn't have to change parameters below this line
ZENOSS_API_URL="${ZENOSS_URL}${ZENOSS_API_PATH}"
ZENOSS_MESSAGE_BEGIN="'{\"action\":\"EventsRouter\",\"method\":\"add_event\",\"data\":[{"
ZENOSS_MESSAGE_END="}],\"type\":\"rpc\",\"tid\":1}'"

SCRIPT_DIR="$SPLUNK_HOME/bin/scripts"
LOG_FILE="$SPLUNK_HOME/var/log/splunk/zenoss-alert-forward.log"
TMP_DIR="/tmp/zenoss-alert-forward.$$"
CURL_POST_SCRIPT="$TMP_DIR/zenoss-alert-forward-curl-post.$$.sh"

echo "START SCRIPT"
echo "--------------------------------------------------------------------------------"
/bin/date

# create tmp dir so that this script can run multiple copies in parallel without collision
mkdir $TMP_DIR

# --------------------------------------------------------------------------------
# *** DETERMINE RESULTS LOCATION AND UNZIP
# --------------------------------------------------------------------------------
RESULTS_FILE=`echo $RESULTS_GZ|awk -F".gz" '{print $1}'`

echo "RESULTS_GZ: $RESULTS_GZ"
echo "RESULTS_FILE: $RESULTS_FILE"

echo "*** cd $TMP_DIR"
cd $TMP_DIR

echo "*** cp $RESULTS_GZ $TMP_DIR"
cp $RESULTS_GZ $TMP_DIR

echo "*** gunzip $RESULTS_FILE"
gunzip $RESULTS_GZ

# --------------------------------------------------------------------------------
# *** echo SCRIPT VARS
# --------------------------------------------------------------------------------

echo "SCRIPT_DIR: $SCRIPT_DIR"
echo "LOG_FILE: $LOG_FILE"
echo "TMP_DIR: $TMP_DIR"

# --------------------------------------------------------------------------------
# *** echo ZENOSS VARS
# --------------------------------------------------------------------------------

#echo "ZENOSS_API_USER: $ZENOSS_API_USER"
#echo "ZENOSS_API_PASS: $ZENOSS_API_PASS"
echo "ZENOSS_URL: $ZENOSS_URL"
echo "ZENOSS_API_PATH: $ZENOSS_API_PATH"
echo "ZENOSS_API_URL: $ZENOSS_API_URL"
echo "EVENT_LIMIT: $EVENT_LIMIT"
echo "SCRIPT_DIR: $SCRIPT_DIR"
echo "LOG_FILE: $LOG_FILE"
echo "TMP_DIR: $TMP_DIR"
echo "ZENOSS_MESSAGE_BEGIN: $ZENOSS_MESSAGE_BEGIN"
echo "ZENOSS_MESSAGE_END: $ZENOSS_MESSAGE_END"

# --------------------------------------------------------------------------------
# *** echo DEFAULT SPLUNK VARS
# --------------------------------------------------------------------------------
echo "SEARCH_COUNT: $SEARCH_COUNT"
#echo "SEARCH_TERMS: $SEARCH_TERMS"
echo "SEARCH_QUERY: $SEARCH_QUERY"
echo "SEARCH_NAME: $SEARCH_NAME"
echo "SEARCH_REASON: $SEARCH_REASON"
echo "SEARCH_URL: $SEARCH_URL"
#echo "SEARCH_TAGS: $SEARCH_TAGS"
echo "SEARCH_PATH: $SEARCH_PATH"

# --------------------------------------------------------------------------------
# *** READ THROUGH RESULTS RECORDS, PARSE & FORWARD ALERTS TO ZENOSS USING CURL
# --------------------------------------------------------------------------------
# this allows us to grab addtional Splunk fields written out from a scheduled alert as a record in the table format
# note: your SPLUNK query must be in the following format: base search | table SEVERITY RESOURCE COMPONENT EVCLASS SUMMARY otherwise you will need to mod this script to use a new format/order

echo "*** cat $RESULTS_FILE"

# enable line below for additional debug
# cat $RESULTS_FILE

# determine field positions from results header
HEADER_RECORD=`cat $RESULTS_FILE | head -1 | sed -e 's/,,/,/g'`
FIELD_01_NAME=`echo $HEADER_RECORD|awk -F"," '{print $1}'|sed -e 's/^"//' -e 's/"$//'`
FIELD_02_NAME=`echo $HEADER_RECORD|awk -F"," '{print $2}'|sed -e 's/^"//' -e 's/"$//'`
FIELD_03_NAME=`echo $HEADER_RECORD|awk -F"," '{print $3}'|sed -e 's/^"//' -e 's/"$//'`
FIELD_04_NAME=`echo $HEADER_RECORD|awk -F"," '{print $4}'|sed -e 's/^"//' -e 's/"$//'`
FIELD_05_NAME=`echo $HEADER_RECORD|awk -F"," '{print $5}'|sed -e 's/^"//' -e 's/"$//'`

echo "HEADER_RECORD: $HEADER_RECORD"
echo "FIELD_01_NAME: $FIELD_01_NAME"
echo "FIELD_02_NAME: $FIELD_02_NAME"
echo "FIELD_03_NAME: $FIELD_03_NAME"
echo "FIELD_04_NAME: $FIELD_04_NAME"
echo "FIELD_05_NAME: $FIELD_05_NAME"

cat $RESULTS_FILE|grep -v "__"|head -$EVENT_LIMIT|while read RECORD
do
        # read through results records, set values and strip off any double quotes
        # skipping every other delimiter since splunk has null values between delimiters in the csv.gz it outputs
        # ie. value1,,value2,,value3,,value4,,value5,
        echo "RECORD: $RECORD"
        # replace double commas with single commas since stats and top commands change the event format
        RECORD=`echo $RECORD | sed -e 's/,,/,/g'`
        echo "RECORD: $RECORD"
        #SEVERITY=`echo $RECORD|awk -F"," '{print $1}'|sed -e 's/^"//' -e 's/"$//'`
        FIELD_01_VALUE=`echo $RECORD|awk -F"," '{print $1}'|sed -e 's/^"//' -e 's/"$//'`
        #RESOURCE=`echo $RECORD|awk -F"," '{print $2}'|sed -e 's/^"//' -e 's/"$//'`
        FIELD_02_VALUE=`echo $RECORD|awk -F"," '{print $2}'|sed -e 's/^"//' -e 's/"$//'`
        #COMPONENT=`echo $RECORD|awk -F"," '{print $3}'|sed -e 's/^"//' -e 's/"$//'`
        FIELD_03_VALUE=`echo $RECORD|awk -F"," '{print $3}'|sed -e 's/^"//' -e 's/"$//'`
        #EVCLASS=`echo $RECORD|awk -F"," '{print $4}'|sed -e 's/^"//' -e 's/"$//'`
        FIELD_04_VALUE=`echo $RECORD|awk -F"," '{print $4}'|sed -e 's/^"//' -e 's/"$//'`
        #SUMMARY=`echo $RECORD|awk -F"," '{print $5}'|sed -e 's/^"//' -e 's/"$//'`
        FIELD_05_VALUE=`echo $RECORD|awk -F"," '{print $5}'|sed -e 's/^"//' -e 's/"$//'`

        case $FIELD_01_NAME in
                severity)
                        SEVERITY=$FIELD_01_VALUE
                        ;;
                resource)
                        RESOURCE=$FIELD_01_VALUE
                        ;;
                component)
                        COMPONENT=$FIELD_01_VALUE
                        ;;
                evclass)
                        EVCLASS=$FIELD_01_VALUE
                        ;;
                summary)
                        SUMMARY=$FIELD_01_VALUE
                        ;;
                *)
                        echo "ERROR: FIELD_01_NAME value of $FIELD_01_NAME not expected!"
                        ;;
        esac

        case $FIELD_02_NAME in
                severity)
                        SEVERITY=$FIELD_02_VALUE
                        ;;
                resource)
                        RESOURCE=$FIELD_02_VALUE
                        ;;
                component)
                        COMPONENT=$FIELD_02_VALUE
                        ;;
                evclass)
                        EVCLASS=$FIELD_02_VALUE
                        ;;
                summary)
                        SUMMARY=$FIELD_02_VALUE
                        ;;
                *)
                        echo "ERROR: FIELD_02_NAME value of $FIELD_02_NAME not expected!"
                        ;;
        esac

        case $FIELD_03_NAME in
                severity)
                        SEVERITY=$FIELD_03_VALUE
                        ;;
                resource)
                        RESOURCE=$FIELD_03_VALUE
                        ;;
                component)
                        COMPONENT=$FIELD_03_VALUE
                        ;;
                evclass)
                        EVCLASS=$FIELD_03_VALUE
                        ;;
                summary)
                        SUMMARY=$FIELD_03_VALUE
                        ;;
                *)
                        echo "ERROR: FIELD_03_NAME value of $FIELD_03_NAME not expected!"
                        ;;
        esac

        case $FIELD_04_NAME in
                severity)
                        SEVERITY=$FIELD_04_VALUE
                        ;;
                resource)
                        RESOURCE=$FIELD_04_VALUE
                        ;;
                component)
                        COMPONENT=$FIELD_04_VALUE
                        ;;
                evclass)
                        EVCLASS=$FIELD_04_VALUE
                        ;;
                summary)
                        SUMMARY=$FIELD_04_VALUE
                        ;;
                *)
                        echo "ERROR: FIELD_04_NAME value of $FIELD_04_NAME not expected!"
                        ;;
        esac

        case $FIELD_05_NAME in
                severity)
                        SEVERITY=$FIELD_05_VALUE
                        ;;
                resource)
                        RESOURCE=$FIELD_05_VALUE
                        ;;
                component)
                        COMPONENT=$FIELD_05_VALUE
                        ;;
                evclass)
                        EVCLASS=$FIELD_05_VALUE
                        ;;
                summary)
                        SUMMARY=$FIELD_05_VALUE
                        ;;
                *)
                        echo "ERROR: FIELD_05_NAME value of $FIELD_05_NAME not expected!"
                        ;;
        esac

        # check for null values and set to "null"

        if [ "$SEVERITY" = "" ]
        then
                SEVERITY="null"
        fi

        if [ "$RESOURCE" = "" ]
        then
                RESOURCE="null"
        fi

        if [ "$COMPONENT" = "" ]
        then
                COMPONENT="null"
        fi

        if [ "$EVCLASS" = "" ]
        then
                EVCLASS="null"
        fi

        if [ "$SUMMARY" = "" ]
        then
                SUMMARY="null"
        fi

        # generate MSG_ID
        MSG_ID_DATE=`date '+%Y-%m-%d.%H.%M.%S'`
        MSG_ID="$MSG_ID_DATE.$$"

        # append MSG_ID to summary for tracing - uncomment line below if you want a unique msg_id to appear in the zenoss summary - makes it simple to trace the message back to the Splunk logs
        #SUMMARY="$SUMMARY msg_id: $MSG_ID"

        echo "MSG_ID: $MSG_ID"
        echo "SEVERITY: $SEVERITY" 
        echo "RESOURCE: $RESOURCE" 
        echo "COMPONENT: $COMPONENT" 
        echo "EVCLASS: $EVCLASS" 
        echo "SUMMARY: $SUMMARY" 

        if [ "$EVCLASS" != "/App/Splunk" ]
        then
                echo "ERROR: EVCLASS not parsed correctly and/or not set to /App/Splunk - This will result in an invalid message being pased to Zenoss!"
        fi

        # NOTE: zenoss does not seem to like the field name resource, although the zenoss UI shows resource, so changed the field name to device before sending the message
        #ZENOSS_MESSAGE_DATA="\"severity\":\"${SEVERITY}\",\"resource\":\"${RESOURCE}\",\"component\":\"${COMPONENT}\",\"evclass\":\"${EVCLASS}\",\"summary\":\"${SUMMARY}\",\"evclasskey\":\"\""
        ZENOSS_MESSAGE_DATA="\"severity\":\"${SEVERITY}\",\"device\":\"${RESOURCE}\",\"component\":\"${COMPONENT}\",\"evclass\":\"${EVCLASS}\",\"summary\":\"${SUMMARY}\",\"evclasskey\":\"\""
        echo "ZENOSS_MESSAGE_DATA: $ZENOSS_MESSAGE_DATA"

        ZENOSS_MESSAGE="$ZENOSS_MESSAGE_BEGIN$ZENOSS_MESSAGE_DATA$ZENOSS_MESSAGE_END"
        echo "ZENOSS_MESSAGE: $ZENOSS_MESSAGE"

        #CURL_COMMAND="curl -v -k -1 -i -u $ZENOSS_API_USER:$ZENOSS_API_PASS -X POST -H \"Content-Type:application/json\" -d"
        #CURL_COMMAND="curl -k -1 -i -u $ZENOSS_API_USER:$ZENOSS_API_PASS -X POST -H \"Content-Type:application/json\" -d"
        CURL_COMMAND="curl -k -1 -u $ZENOSS_API_USER:$ZENOSS_API_PASS -X POST -H \"Content-Type:application/json\" -d"
        echo "CURL_COMMAND: $CURL_COMMAND"

        POST_COMMAND="$CURL_COMMAND $ZENOSS_MESSAGE $ZENOSS_API_URL"
        echo "POST_COMMAND: $POST_COMMAND"

        # had some trouble with the shell/curl properly passing arguments to zenoss, opted to build and run a dynamic script
        # create script 
        echo "$POST_COMMAND" > $CURL_POST_SCRIPT

        # set execute perms on script
        chmod 700 $CURL_POST_SCRIPT

        # execute script
        $CURL_POST_SCRIPT 2>&1

        # remove script after execution
        rm -f $CURL_POST_SCRIPT
done

# --------------------------------------------------------------------------------
# *** CLEANUP
# --------------------------------------------------------------------------------

# remove tmp files generated by the script
echo
echo "*** rm -rf $TMP_DIR"
rm -rf $TMP_DIR

/bin/date
echo "--------------------------------------------------------------------------------"
echo "END SCRIPT"

exit
0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!