Splunk Search

Need Help with a Splunk query

phularah
Communicator

So, I have data like this after I ran a query. 

phularah_1-1715356433010.png

For each aggregator, if the aggregator_status is Error and before15 minutes, the aggregator_status becomes Up, alert should not run. But, if the aggregator_status is still Error or no new event comes, alert should trigger. The Time field is epoch time which I am thinking can be used to find difference in Up and Error status times.

How do I create such a query for the alert? I am thinking of using foreach command or some sort of streamstats, but I am unable to resolve this issue. The alert needs to run once every 24 hours.

Labels (3)
0 Karma

PickleRick
SplunkTrust
SplunkTrust

It might be doable with the transaction command but it's usually not a good idea (transaction is a relatively "heavy" command and has its limitations).

I'd go with streamstats and reset_before, reset_after and time_window options. (can't give you a ready-made answer at the moment since I'm away from my Splunk environment but that's the way I'd try)

0 Karma

phularah
Communicator

I tried something like this 

index=abc ("Aggregator * is Error" OR "Aggregator * is Up") NJ12GC102
| rex field=_raw "Aggregator\s(?<aggregator>[^\s]+)\sis\s(?<aggregator_status>\w+)\s"
| streamstats current=t global=f window=2 range(_time) as time_diff by aggregator,aggregator_status
| streamstats current=t global=f window=2 range(_time) as time_diff2 by aggregator
| table _time aggregator aggregator_status time_diff time_diff2
|



But the output is now what I needed. For that I would need to change the window=2, but it brings more issues.

phularah_1-1715358409563.png

 

 

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

Try starting with something like this

| streamstats values(aggregator_status) as previous_aggregator_status by aggregator window=1 current=f global=f
| eval changetime=if((aggregator_status="Up" and previous_aggregator_status="Error") or (aggregator_status="Error" and previous_aggregator_status="Up"),_time,null())
| where isnotnull(changetime)
| streamstats current=t global=f window=2 range(_time) as time_diff2 by aggregator
| where aggregator_status="Error"
0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Announcing Modern Navigation: A New Era of Splunk User Experience

We are excited to introduce the Modern Navigation feature in the Splunk Platform, available to both cloud and ...

Modernize your Splunk Apps – Introducing Python 3.13 in Splunk

We are excited to announce that the upcoming releases of Splunk Enterprise 10.2.x and Splunk Cloud Platform ...

Step into “Hunt the Insider: An Splunk ES Premier Mystery” to catch a cybercriminal ...

After a whole week of being on call, you fell asleep on your keyboard, and you hit a sequence of buttons that ...