Alerting

Problem with lookup for disabling alerts during maintenance

jmoeller
New Member

Sorry for the simple question, I am new to the Splunk world....

I have a CSV loaded (StandardMaintenance.csv) which has two rows

UnderMaintenance
NO

I want to add a check to each alert so that they will not fire during maintenance.

Here is my code...

....query goes here...
| lookup StandardMaintenance.csv UnderMaintenance
| search NOT UnderMaintenance="NO"

What am I doing wrong or how better might I accomplish this?

Thanks.

0 Karma

jmoeller
New Member

Thanks for the response. The CVS file only consists of the one field. I want maintenance to be either on or off. If off, run query, if on, don't run query. I am not marking individual hosts, it is all or nothing.

Thanks

0 Karma

fferozbasha
Explorer
  1. The Inputlookup command supports where condition which means the better way to filter down the search will be

| inputlookup StandardMaintenance WHERE UnderMaintenance="Yes"

  1. What columns do you expect out of the lookup and added in the filter in base search? Ex - If it is to find the list of Hosts that are not under maintenance and consider them for alerting, sample query can be -

index=os sourcetype=vmstat NOT [ | inputlookup StandardMaintenance WHERE UnderMaintenance="Yes" | fields host | table host ]
| fields memUsedPct, host
| stats avg(memUsedPct) as avgMemUsed by host
| where avgMemUsed > 75
| table host

0 Karma

jmoeller
New Member

Thanks. I have no additional information in the StandardMaintenance table. I simply want to use the table as a check... If UnderMaintainance is NO, execute query (thus generate an alert).

To use your example, I want to run the index=os sourcetype=vmstat query only when UnderMaintainence is "NO"

I know I should be able to deduce it from what you provided, but I am still struggling. Any additional input is appreciated.

0 Karma

nagarjuna559
Explorer

| lookup StandardMaintenance.csv UnderMaintenance
| where UnderMaintenance="NO"

fields values you use after where are the case sensitive,

0 Karma

nagarjuna559
Explorer

| lookup StandardMaintenance.csv

| where UnderMaintenance="NO"

0 Karma

fferozbasha
Explorer

The query that i have provided should do the same. It will run search a Index=os ONLY if there any hosts tagged as UnderMaintenance=Yes.

0 Karma

renjith_nair
Legend

What are you looking up against i.e. what's matching criteria in search and lookup?
Also in the above search if you want only those events which is not under maintenance , you should use

| search UnderMaintenance="NO"

---
What goes around comes around. If it helps, hit it with Karma 🙂
0 Karma

nvanderwalt_spl
Splunk Employee
Splunk Employee

Do you want to put all the things in downtime?
If you had only a few hosts in the lookup at any time, you could do something like this:

search string NOT [|inputlookup hosts_in_maint.csv | table  host| format] 
0 Karma

jmoeller
New Member

I am sorry I wasn't clear. I want it to function as a simple on/off. If maintenance is ON, don't run the search. If maintenance is off, run it normally.

Thanks for your input.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Break the Build: Inside the KubeDoom Lounge at .conf26

    You step up to the machine. The pixelated corridors of a certain 1993 FPS load in front of you, EMP Pulse ...

Splunk Auto Ingestion Parallel Pipeline Scaling

Why this feature matters Many Splunk environments experience ingestion pressure long before the host is fully ...

Best Practices: Splunk auto adjust pipeline queue

When you enable autoAdjustQueue in Splunk, maxSize should be understood as the queue size Splunk starts with ...