All Apps and Add-ons
Highlighted

Splunk Alert

Contributor

Dear All,

We are using splunk for monitoring purpose. We are pulling data from multiple application We want fire an alert suppose if i wont get the one statement in particular log file. I get a one event at every two hour so whenever i get that event i get a alert. But my requirement is different if i wont get this event in 2 hours then i have to get a alert . Can anybody help me how to get this done?

Thanks

Gajanan Hiroji

0 Karma
Highlighted

Re: Splunk Alert

SplunkTrust
SplunkTrust

You could have your alert trigger whenever there are zero results. For example, run this realtime with a two-hour window:

your search terms | stats count

Configure the alert's trigger condition to fire when count is zero. Make sure to also set throttling.

View solution in original post

Highlighted

Re: Splunk Alert

Contributor

Thanks for your reply but when there is alert we can schedule for 2 hours in splunk.?

0 Karma
Highlighted

Re: Splunk Alert

SplunkTrust
SplunkTrust

I don't think I understand that question, please elaborate on what you're asking.

0 Karma
Highlighted

Re: Splunk Alert

Contributor

Whenever i am scheduling alert the option for time range is 1) Every hour 2) every day kind of thing but there is no every two hour option where i have to do this conf files?

0 Karma
Highlighted

Re: Splunk Alert

SplunkTrust
SplunkTrust

Ah. Select cron schedule and enter this:

0 */2 * * *

That'll run at 00:00, 02:00, 04:00, etc.

If there's an event at 00:30 and one at 03:00 that's more than two hours... should your alert fire in this case? Scheduling this with a two-hour time range every two hours would not fire because there was an event in each two-hour time range despite the difference between the events being greater than two hours.

0 Karma
Highlighted

Re: Splunk Alert

Contributor

Yes if the event doesn't come in 2 hour then i have fire a alert.

0 Karma
Highlighted

Re: Splunk Alert

Communicator

Also, you could evaluate the search, by calculating a "gap". Then alert when your gap is too long.
This is how I did it:

| stats max(_time) As LatestTime by appserver | eval gap=(now()-LatestTime) | eval appserver="WebSphere" | sort num(gap) D | head 1 | rangemap field=gap low=0-300 default=severe

Highlighted

Re: Splunk Alert

SplunkTrust
SplunkTrust

Note, you need to use a time range longer than two hours for this to work... Long enough to be sure that the most recent event is still visible but older than two hours when running the alert.

Highlighted

Re: Splunk Alert

Contributor

Thanks Renems. Can you elaborate this i am not able to get the | eval appserver="WebSphere" | sort num(gap) D | head 1 | rangemap field=gap low=0-300 default=severe

0 Karma