Splunk Search

Please help to setup alert?

bhaskar5428
Explorer
index=* ("ORC from FCS completed" OR "ORC from SDS completed." OR "ORC from ROUTER completed") namespace IN ("dk1692-b","dk1399-b","dk1371-b")
| eval dk1692=if(searchmatch("\"ORC from ROUTER completed\" namespace=dk1692-b"),1,0)
| eval dk1399=if(searchmatch("\"ORC from SDS completed\" namespace=dk1399-b"),1,0)
| eval dk1371=if(searchmatch("\"ORC from FCS completed\" namespace=dk1371-b"),1,0)
| stats sum(dk*) as dk*
| search dk1692>90 OR dk1399>60 OR dk1371>60

 

Am getting attached output 

 

bhaskar5428_0-1677482540712.png

Problem statement ,
i would like setup alert where ever specific namespace **bleep** goes below its threshold which mention in search query. 

Labels (1)
0 Karma

bhaskar5428
Explorer

Sorry , please check latest screen shot , in this wherever  namespace value less than threashhod it shoud throw email alert 

For example - dk1399 is 45 which is less than 60

bhaskar5428_0-1677495212425.png

 

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

OK, so the fact that this search returns any rows means that at least one of the namespaces is below its threshold. You can use this fact as the trigger in your alert

ITWhisperer_0-1677495642402.png

 

0 Karma

bhaskar5428
Explorer

Yes but there are three namespaces with different threshould 

Do i have to setup 3 different alerts by using less than trigger ??

 

bhaskar5428_0-1677499385972.png

 

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

It depends on your requirement. You could have a single alert for any of them being below the threshold, or if you had them in different rows, you could still have a single alert that would trigger for each result.

| where dk1692<90 OR dk1399<60 OR dk1371<60
| eval _time=now()
| untable _time namespace count
| where (namespace="dk1692" and count<90) OR (namespace="dk1399" and count<60) OR (namespace="dk1371" and count<60)
0 Karma

bhaskar5428
Explorer

index=* ("ORC from FCS completed" OR "ORC from SDS completed." OR "ORC from ROUTER completed") namespace IN ("dk1692-b","dk1399-b","dk1371-b")
| eval dk1692=if(searchmatch("\"ORC from ROUTER completed\" namespace=dk1692-b"),1,0)
| eval dk1399=if(searchmatch("\"ORC from SDS completed\" namespace=dk1399-b"),1,0)
| eval dk1371=if(searchmatch("\"ORC from FCS completed\" namespace=dk1371-b"),1,0)
| stats sum(dk*) as dk*
| search dk1692<90 OR dk1399<60 OR dk1371<60
| eval _time=now()
| untable _time namespace count
| where (namespace="dk1692" and count<90) OR (namespace="dk1399" and count<60) OR (namespace="dk1371" and count<60)

-------------------------------------------------------------------------------------------------------------

this query good as per my requirement , however how can i setup alert with trigger 

Because its giving one row and count.

for setting up condition can i use 
number of results - is greater than = 0

*** So whenever there is record in above query it will send alert. 

 

 

bhaskar5428_0-1677559034501.png

 

bhaskar5428_1-1677559060821.png

 

 

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

If you also set "For each result", you will get a triggered action for each row i.e. each namespace that is below its threshold

ITWhisperer_0-1677571110575.png

You would need to set an action to be triggered, e.g. an email being sent for each trigger.

0 Karma

bhaskar5428
Explorer

I would like to use throttle option 

But when am using each result option for trigger its asking field name suppressing.

 

how to manage that ? 

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

You could try 

namespace = $result.namespace$
0 Karma

bhaskar5428
Explorer

you mean i need to add value

namespace = $result.namespace$

into Suppress results containing field value ??

 

please confirm 

 
Tags (1)
0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

That is what I mean - give it a try 😀

0 Karma

bhaskar5428
Explorer

Thank you so much

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

Your search is looking for values greater than specific values. Do these indicate that the namespace has gone below its threshold? If so, just copy the search into an alert and trigger when there are greater than zero results.

0 Karma

bhaskar5428
Explorer

index=* ("ORC from FCS completed" OR "ORC from SDS completed." OR "ORC from ROUTER completed" OR "ORC from EMS completed" OR "ORC from TDS completed") namespace IN ("dk1692-b","dk1399-b","dk1371-b","dk1398-b","dk1400-c")
| eval dk1692=if(searchmatch("\"ORC from ROUTER completed\" namespace=dk1692-b"),1,0)
| eval dk1399=if(searchmatch("\"ORC from SDS completed\" namespace=dk1399-b"),1,0)
| eval dk1371=if(searchmatch("\"ORC from FCS completed\" namespace=dk1371-b"),1,0)
| eval dk1398=if(searchmatch("\"ORC from EMS completed\" namespace=dk1398-b"),1,0)
| eval dk1400=if(searchmatch("\"ORC from TDS completed\" namespace=dk1400-c"),1,0)
| stats sum(dk*) as dk*
| search dk1692<90 OR dk1399<60 OR dk1371<60 OR dk1398<60 OR dk1400<60
| eval _time=now()
| untable _time namespace count
| where (namespace="dk1692" and count<90) OR (namespace="dk1399" and count<60) OR (namespace="dk1371" and count<60) OR (namespace="dk1398" and count<60) OR (namespace="dk1400" and count<60)

 

----------------------------------------------------------------------------------------------------------------

Thanks in above query , i would like to add one more column which says which namespace is below threshold 

example if namespace dk1399-b below 60 it column is should show name SDS 

 

0 Karma
Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...