Deployment Architecture
Highlighted

Does someone have a shell script/command to check the status and health of search head and indexer clusters?s

Builder

I am looking into writing something that will check the health of my indexer and search head clusters. This will eventually be put into an enterprise monitoring tool and alerts sent out if there is an "issue". I know you can run splunk show cluster-status etc... We are running this all on Linux and I am thinking that someone may have a shell script that can be used to verify health of this environment.

Any suggestions?

0 Karma
Highlighted

Re: Does someone have a shell script/command to check the status and health of search head and indexer clusters?s

SplunkTrust
SplunkTrust

Have you seen Splunk Health Overview app?? This does provide health of Splunk servers (whichever are added as search peer) using REST API endpoints using search.
https://splunkbase.splunk.com/app/1919/

0 Karma
Highlighted

Re: Does someone have a shell script/command to check the status and health of search head and indexer clusters?s

Esteemed Legend

For a very gross view, you can use $SPLUNK_HOME/bin/splunk status.

0 Karma
Highlighted

Re: Does someone have a shell script/command to check the status and health of search head and indexer clusters?s

Champion

I use the bash alias command called splunkstatus using the following command and script.

alias splunkstatus=' "ssh <search_head> "sudo -u <user_running_splunk> /opt/splunk/bin/splunk show shcluster-status -auth <user>:<password>" | ~/statusparse.sh'

SCRIPT:

#!/bin/bash
count=1;
output="$(/usr/bin/awk '{print $3}')"
while read line; 
do
  if [[ $line =~ ^https://<prefix>*. ]]; then 
    echo "Search Captian: $line";
  fi;
  if [[ $line =~ ^<search_prefix>*. ]]; then
     search="$line";
   fi;
  if [[ $line =~ ^Down|Up|Restarting ]]; then
    search="$count $search $line";
    echo "$search";
    ((count++))
  fi;
done <<< "$output"

You'll need to mod it a bit.

0 Karma