Alerting

Trigger specific custom alert

alekseisaiko
Path Finder

Hi there!
I'm using this query

index="dev" |eval raw_len=len(_raw) | eval raw_len_gb = raw_len/1024/1024/1024 | stats sum(raw_len_gb) as GB by kubernetes_namespace | bin _time span=1d

To get the amount of received data from K8S logs, and I want to trigger alert using this query, if the amount will be more than 0.5 gig a day.
How to define it in alert?
Right now, when I'm defining alert that triggers when - "eval raw_len_gb > 0.5", it's not triggered

Thanks,

ALeksei

Tags (2)
0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

Add | where raw_len_gb > 0.5 to the end of your query and change the alert to trigger if the number of results is not zero.

---
If this reply helps you, Karma would be appreciated.

View solution in original post

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Add | where raw_len_gb > 0.5 to the end of your query and change the alert to trigger if the number of results is not zero.

---
If this reply helps you, Karma would be appreciated.
0 Karma

alekseisaiko
Path Finder

Thanks for you answer!
Right now, I want to test that this query works.

If I'm running my query (without the | where raw_len_gb > 0.5) for last 15 mins, I'm getting results, with 0.029 GB for example in one of the results.
But if I want to check the same query with | where raw_len_gb > 0.01 just to see that it will show results, it doesn't. Though it must show the one with 0.029 GB and more.

0 Karma

alekseisaiko
Path Finder

Ok, it worked liked this -
index="dev" |eval raw_len=len(_raw) | eval raw_len_gb = raw_len/1024/1024/1024 | stats sum(raw_len_gb) as GB by kubernetes_namespace | bin _time span=1d | where GB > 0.0004
And I'm getting relevant results now, but still , trigger is not working from some reason

0 Karma

alekseisaiko
Path Finder

My bad, it worked! 🙂
Thanks

0 Karma
Get Updates on the Splunk Community!

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...