Alerting

Base line alerts in splunk

smaran06
Path Finder

Hi Team,

I have a requirement in splunk, where if instance count went down it should alert. For example if I have 10 instances running in a application and in which 4 went down, then splunk should be able to compare previous results and present results and trigger the alert as its less than 10.

Basically, I want base line alerts, where it should compare previous values to current and if its not same it should, Please let me know how this can be done.

Tags (2)
0 Karma

DalJeanis
Legend

There are a couple of different strategies.

First, you can create a search that determines the counts at the two different points in time, and compares the two numbers.

Second, you can create a periodic search that calculates the counts at the current moment and writes that number to a summary index. Then, you compare the last record on the smmary index to the prior record, and alert if the number drops.

HattrickNZ
Motivator

so 1/ and 2/ are the same except 2/ uses a summary index as its baseline, whereas 1 just uses a search with the time controlled by earliest = and latest=?

DalJeanis
Legend

@HattrickNZ - Yep, you can use a single search that checks for two points in the past, or you can create a summary index or lookup table and use that. There are other ways, but those are pretty straightforward.

0 Karma

HattrickNZ
Motivator

tks, what are the other ways? I'd like to know for something i am working on at the minute. Because them 2 options you mention are pretty limited to what can be put in the search e.g. averages, maxes or mins of certain periods. Maybe I am looking for some more advanced type stats analysis/baselining..

0 Karma
Get Updates on the Splunk Community!

Detecting Remote Code Executions With the Splunk Threat Research Team

WATCH NOWRemote code execution (RCE) vulnerabilities pose a significant risk to organizations. If exploited, ...

Enter the Splunk Community Dashboard Challenge for Your Chance to Win!

The Splunk Community Dashboard Challenge is underway! This is your chance to showcase your skills in creating ...

.conf24 | Session Scheduler is Live!!

.conf24 is happening June 11 - 14 in Las Vegas, and we are thrilled to announce that the conference catalog ...