Splunk Search

How to edit my search to alert if there are 300,000 less events within a 1 hour period?

mcbradford
Contributor

I have the following search:

| metadata type=sourcetypes| stats sum(totalCount)

I want to be alerted if within a one hour period, there are 300k less events. I was using the drops by, but with the search above, there is only one event (the sum of the total count).

Thoughts?

Tags (3)
0 Karma

LukeMurphey
Champion

You can setup a search that returns nothing unless the count if less than 300000 and then use an alert. You can easily make this in the Manager UI too.

Here are some details:

Make the search logic

Create a search that return nothing unless the count hits the threshold.

| metadata type=sourcetypes| stats sum(totalCount) as count | where count < 300000

Schedule search

Schedule the search to run every hour and over a one hour period. This will generate an alert every hour but you could actually have it run more frequently.

Setup alert

Setup the alert to only trigger if at least on result is generated. That way, you won't get an alert unless the count is below your threshold.

Sample search

Here is my version of the search (not extensively tested though):

[Alert: event count low]
alert.digest_mode = True
alert.suppress = 0
alert.track = 0
auto_summarize.dispatch.earliest_time = -1d@h
counttype = number of events
quantity = 0
relation = greater than
cron_schedule = 0 * * * *
dispatch.earliest_time = -60m
dispatch.latest_time = now
enableSched = 1
search = | metadata type=sourcetypes| stats sum(totalCount) as count | where count < 300000
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...