Base line alerts in splunk

[smaran06]

Hi Team,

I have a requirement in splunk, where if instance count went down it should alert. For example if I have 10 instances running in a application and in which 4 went down, then splunk should be able to compare previous results and present results and trigger the alert as its less than 10.

Basically, I want base line alerts, where it should compare previous values to current and if its not same it should, Please let me know how this can be done.

[DalJeanis]

There are a couple of different strategies.

First, you can create a search that determines the counts at the two different points in time, and compares the two numbers.

Second, you can create a periodic search that calculates the counts at the current moment and writes that number to a summary index. Then, you compare the last record on the smmary index to the prior record, and alert if the number drops.

[HattrickNZ]

so 1/ and 2/ are the same except 2/ uses a summary index as its baseline, whereas 1 just uses a search with the time controlled by earliest = and latest=?

[DalJeanis]

@HattrickNZ - Yep, you can use a single search that checks for two points in the past, or you can create a summary index or lookup table and use that. There are other ways, but those are pretty straightforward.

[HattrickNZ]

tks, what are the other ways? I'd like to know for something i am working on at the minute. Because them 2 options you mention are pretty limited to what can be put in the search e.g. averages, maxes or mins of certain periods. Maybe I am looking for some more advanced type stats analysis/baselining..