topic Re: Base line alerts in splunk in Alerting

Base line alerts in splunk

smaran06 — Mon, 11 Dec 2017 23:14:00 GMT

Hi Team,

I have a requirement in splunk, where if instance count went down it should alert. For example if I have 10 instances running in a application and in which 4 went down, then splunk should be able to compare previous results and present results and trigger the alert as its less than 10.

Basically, I want base line alerts, where it should compare previous values to current and if its not same it should, Please let me know how this can be done.

Re: Base line alerts in splunk

DalJeanis — Mon, 11 Dec 2017 23:43:43 GMT

There are a couple of different strategies.

First, you can create a search that determines the counts at the two different points in time, and compares the two numbers.

Second, you can create a periodic search that calculates the counts at the current moment and writes that number to a summary index. Then, you compare the last record on the smmary index to the prior record, and alert if the number drops.

Re: Base line alerts in splunk

HattrickNZ — Mon, 21 May 2018 22:01:11 GMT

so 1/ and 2/ are the same except 2/ uses a summary index as its baseline, whereas 1 just uses a search with the time controlled by earliest = and latest=?

Re: Base line alerts in splunk

DalJeanis — Tue, 22 May 2018 02:47:56 GMT

@HattrickNZ - Yep, you can use a single search that checks for two points in the past, or you can create a summary index or lookup table and use that. There are other ways, but those are pretty straightforward.

Re: Base line alerts in splunk

HattrickNZ — Tue, 22 May 2018 04:03:57 GMT

tks, what are the other ways? I'd like to know for something i am working on at the minute. Because them 2 options you mention are pretty limited to what can be put in the search e.g. averages, maxes or mins of certain periods. Maybe I am looking for some more advanced type stats analysis/baselining..