Splunk Search

How do you use multiple thresholds from a timechart for a single alert for an arbitrary number of devices?

michaelsplunk1
Path Finder
_timedevice1_avgdevice2_avgdevice3_avgdevice4_avg
2022-04-07 00:003431122
2022-04-07 01:0021764187
2022-04-07 02:002183232
2022-04-07 03:001233654
2022-04-07 04:00782143
2022-04-07 05:001131721
2022-04-07 06:0019121916
2022-04-07 07:0015101219
2022-04-07 08:0042196

 

I have a table of averages for an arbitrary number of arbitrary devices as shown above. How do I use these averages as thresholds for alerts about these devices? I'm trying to have a search that runs every 15 minutes to check which devices have exceeded these averages.

For example, if a search is run at 06:45, and returns that device1 has a count of 10, device2 has a count of 15, device3 has a count of 21, and device 4 has a count of 2, send an alert that says device2 and device3 have exceeded their averages listed for the 06:00 hour (i.e., 12 and 19, respectively).

Labels (1)
Tags (3)
0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

Here's an untested idea.  Round the runtime of the search to the beginning of the hour.  Look up the result in the _time field of the averages table, returning all device thresholds for that hour.  Compare the calculated average to the threshold and trigger an alert if the result count is not zero.

```Round the runtime of the search to the beginning of the hour. ```
| eval lookupTime=relative_time(_time, "@h")
```Look up the result in the _time field of the averages table, returning all device thresholds for that hour.  ```
| lookup averages.csv _time OUTPUT device1_avg AS device1_thresh, device2_avg AS device2_thresh, device3_avg AS device3_thresh, device4_avg AS device4_thresh
```Compare the calculated average to the threshold```
| where (device1_avg > device1_thresh OR device2_avg > device2_thresh OR device3_avg > device3_thresh OR device4_avg > device4_thresh)

 

---
If this reply helps you, Karma would be appreciated.

View solution in original post

richgalloway
SplunkTrust
SplunkTrust

Here's an untested idea.  Round the runtime of the search to the beginning of the hour.  Look up the result in the _time field of the averages table, returning all device thresholds for that hour.  Compare the calculated average to the threshold and trigger an alert if the result count is not zero.

```Round the runtime of the search to the beginning of the hour. ```
| eval lookupTime=relative_time(_time, "@h")
```Look up the result in the _time field of the averages table, returning all device thresholds for that hour.  ```
| lookup averages.csv _time OUTPUT device1_avg AS device1_thresh, device2_avg AS device2_thresh, device3_avg AS device3_thresh, device4_avg AS device4_thresh
```Compare the calculated average to the threshold```
| where (device1_avg > device1_thresh OR device2_avg > device2_thresh OR device3_avg > device3_thresh OR device4_avg > device4_thresh)

 

---
If this reply helps you, Karma would be appreciated.
Get Updates on the Splunk Community!

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...

Built-in Service Level Objectives Management to Bridge the Gap Between Service & ...

Wednesday, May 29, 2024  |  11AM PST / 2PM ESTRegister now and join us to learn more about how you can ...

Get Your Exclusive Splunk Certified Cybersecurity Defense Engineer Certification at ...

We’re excited to announce a new Splunk certification exam being released at .conf24! If you’re headed to Vegas ...