Alerting
Highlighted

Splunk conditional alert

Contributor

Hi Splunkers,

I have a saved search which returns the status of certain services in our infrastructure.
It returns in this format:


Servicename | Status
Service 1 | OK
Service 2 | OK
Service 3 | OK
Service 4 | Error 204
Service 5 | Error 400

I want the search to trigger an hourly alert if any of the systems aren't "OK". I'm using the following custom condition:

where "Service 1"!="OK" OR "Service 2"!="OK" OR "Service 3"!="OK" OR "Service 4"!="OK" OR "Service 5"!="OK"

The problem I'm having, is that the alert is triggering every hour when the service status is "OK". It doesn't seem to be accepting the conditions.

Can anyone see something wrong with my conditions? I can't find much in the documentation to go on with..

0 Karma
Highlighted

Re: Splunk conditional alert

Contributor

I don't have access to my Splunk Server right now. However, I would use a nested 'IF' command to check the conditions and raise the alert.

0 Karma
Highlighted

Re: Splunk conditional alert

Contributor

Hi, thanks for the suggestion. I tried this, but it didn't make a difference. I was using the transpose command to make the table look neater, but this seems to cause problems when using it in conjunction with alerts. I removed the transpose command and it started to work, but the e-mail from the alert isn't in the format I showed in the question. I suppose it will have to do!


Service 1 | Service 2 | Service 3 | Service 4 | Service 5
OK | Error 400 | OK | OK | OK

0 Karma
Highlighted

Re: Splunk conditional alert

Contributor

Can you share / email me the search string? I now have access a Test Splunk instance.

0 Karma
Highlighted

Re: Splunk conditional alert

New Member

Hi

I need to setup a condition to an existing alert where the alert shouldn't trigger on next day of Bank Holidays. How do i set that condition .The Alert looks for specific file on a server

0 Karma
Highlighted

Re: Splunk conditional alert

Splunk Employee
Splunk Employee

Hi @nages,

This post is a few years old so it may not garner the type of activity that you're seeking. If you need some help, I would suggest posting a new question.

Or, if you want to try to get some immediate help for your question, you should join the 1300+ Splunk users in our public Slack chat. People ask each other for immediate help on there daily. You can share your question there to see if anyone can take a stab at it.

You first have to request access through www.splunk402.com/chat. Fill out the form, and once you receive the approval email from our Community Manager (usually the approval process takes a couple days), you can access Slack.com and ask for help in the #general channel.

0 Karma