Splunk Search

Search for peers with status=down

pc1
Path Finder

What search can I do to find peers with status=down. Looking to form an alert when this happens but can't find it within a search. 

Labels (1)
0 Karma
1 Solution

somesoni2
Revered Legend

Splunk Monitoring console (formally known as DMC) has alert "DMC Alert - Search Peer Not Responding" which does the same thing. It basically runs following search:

 

| rest splunk_server=local /services/search/distributed/peers/
| where status!="Up"
| fields peerName, status
| rename peerName as Instance, status as Status

 

View solution in original post

somesoni2
Revered Legend

Splunk Monitoring console (formally known as DMC) has alert "DMC Alert - Search Peer Not Responding" which does the same thing. It basically runs following search:

 

| rest splunk_server=local /services/search/distributed/peers/
| where status!="Up"
| fields peerName, status
| rename peerName as Instance, status as Status

 

pc1
Path Finder

Yup, found this preexisting alert and was able to edit the Actions on it to integrate with the Slack Notifications add-on. Runs every 5 minutes to check if the server is down so this works perfectly for me. Thanks for the help!

0 Karma

Stefanie
Builder

Are you looking for hosts with forwarders installed that havent reported to Splunk in some time?

You can use the Monitoring Console to view that.  To view the missing hosts, you can click on the Forwarders tab and then Forwarders: Deployment.

For an alert, go to the Monitoring Console -> Settings -> Alerts Setup. There is an alert named DMC Alert - Missing Forwarders.

Note: A forwarder shows a status of "missing" if it has not connected to indexers within 15 minutes

0 Karma
Get Updates on the Splunk Community!

Using Machine Learning for Hunting Security Threats

WATCH NOW Seeing the exponential hike in global cyber threat spectrum, organizations are now striving more for ...

New Learning Videos on Topics Most Requested by You! Plus This Month’s New Splunk ...

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...

How I Instrumented a Rust Application Without Knowing Rust

As a technical writer, I often have to edit or create code snippets for Splunk's distributions of ...