Splunk Search

How to edit my search to alert if there are 300,000 less events within a 1 hour period?

mcbradford
Contributor

I have the following search:

| metadata type=sourcetypes| stats sum(totalCount)

I want to be alerted if within a one hour period, there are 300k less events. I was using the drops by, but with the search above, there is only one event (the sum of the total count).

Thoughts?

Tags (3)
0 Karma

LukeMurphey
Champion

You can setup a search that returns nothing unless the count if less than 300000 and then use an alert. You can easily make this in the Manager UI too.

Here are some details:

Make the search logic

Create a search that return nothing unless the count hits the threshold.

| metadata type=sourcetypes| stats sum(totalCount) as count | where count < 300000

Schedule search

Schedule the search to run every hour and over a one hour period. This will generate an alert every hour but you could actually have it run more frequently.

Setup alert

Setup the alert to only trigger if at least on result is generated. That way, you won't get an alert unless the count is below your threshold.

Sample search

Here is my version of the search (not extensively tested though):

[Alert: event count low]
alert.digest_mode = True
alert.suppress = 0
alert.track = 0
auto_summarize.dispatch.earliest_time = -1d@h
counttype = number of events
quantity = 0
relation = greater than
cron_schedule = 0 * * * *
dispatch.earliest_time = -60m
dispatch.latest_time = now
enableSched = 1
search = | metadata type=sourcetypes| stats sum(totalCount) as count | where count < 300000
0 Karma
Get Updates on the Splunk Community!

Splunk Decoded: Service Maps vs Service Analyzer Tree View vs Flow Maps

It’s Monday morning, and your phone is buzzing with alert escalations – your customer-facing portal is running ...

What’s New in Splunk Observability – September 2025

What's NewWe are excited to announce the latest enhancements to Splunk Observability, designed to help ITOps ...

Fun with Regular Expression - multiples of nine

Fun with Regular Expression - multiples of nineThis challenge was first posted on Slack #regex channel ...