Alerting
Highlighted

Problem with lookup for disabling alerts during maintenance

New Member

Sorry for the simple question, I am new to the Splunk world....

I have a CSV loaded (StandardMaintenance.csv) which has two rows

UnderMaintenance
NO

I want to add a check to each alert so that they will not fire during maintenance.

Here is my code...

....query goes here...
| lookup StandardMaintenance.csv UnderMaintenance
| search NOT UnderMaintenance="NO"

What am I doing wrong or how better might I accomplish this?

Thanks.

0 Karma
Highlighted

Re: Problem with lookup for disabling alerts during maintenance

Splunk Employee
Splunk Employee

Do you want to put all the things in downtime?
If you had only a few hosts in the lookup at any time, you could do something like this:

search string NOT [|inputlookup hosts_in_maint.csv | table  host| format] 
0 Karma
Highlighted

Re: Problem with lookup for disabling alerts during maintenance

New Member

I am sorry I wasn't clear. I want it to function as a simple on/off. If maintenance is ON, don't run the search. If maintenance is off, run it normally.

Thanks for your input.

0 Karma
Highlighted

Re: Problem with lookup for disabling alerts during maintenance

SplunkTrust
SplunkTrust

What are you looking up against i.e. what's matching criteria in search and lookup?
Also in the above search if you want only those events which is not under maintenance , you should use

| search UnderMaintenance="NO"

0 Karma
Highlighted

Re: Problem with lookup for disabling alerts during maintenance

Explorer
  1. The Inputlookup command supports where condition which means the better way to filter down the search will be

| inputlookup StandardMaintenance WHERE UnderMaintenance="Yes"

  1. What columns do you expect out of the lookup and added in the filter in base search? Ex - If it is to find the list of Hosts that are not under maintenance and consider them for alerting, sample query can be -

index=os sourcetype=vmstat NOT [ | inputlookup StandardMaintenance WHERE UnderMaintenance="Yes" | fields host | table host ]
| fields memUsedPct, host
| stats avg(memUsedPct) as avgMemUsed by host
| where avgMemUsed > 75
| table host

0 Karma
Highlighted

Re: Problem with lookup for disabling alerts during maintenance

New Member

Thanks. I have no additional information in the StandardMaintenance table. I simply want to use the table as a check... If UnderMaintainance is NO, execute query (thus generate an alert).

To use your example, I want to run the index=os sourcetype=vmstat query only when UnderMaintainence is "NO"

I know I should be able to deduce it from what you provided, but I am still struggling. Any additional input is appreciated.

0 Karma
Highlighted

Re: Problem with lookup for disabling alerts during maintenance

Explorer

The query that i have provided should do the same. It will run search a Index=os ONLY if there any hosts tagged as UnderMaintenance=Yes.

0 Karma
Highlighted

Re: Problem with lookup for disabling alerts during maintenance

Explorer

| lookup StandardMaintenance.csv UnderMaintenance
| where UnderMaintenance="NO"

fields values you use after where are the case sensitive,

0 Karma
Highlighted

Re: Problem with lookup for disabling alerts during maintenance

Explorer

| lookup StandardMaintenance.csv

| where UnderMaintenance="NO"

0 Karma
Highlighted

Re: Problem with lookup for disabling alerts during maintenance

New Member

Thanks for the response. The CVS file only consists of the one field. I want maintenance to be either on or off. If off, run query, if on, don't run query. I am not marking individual hosts, it is all or nothing.

Thanks

0 Karma