Splunk Search

How to trigger an alert if 3 consecutive search results reach a certain threshold?

snehalk
Communicator

Hello All,

I have one requirement where an alert needs to be triggered after three continuous search results reach a certain threshold.

Eg: am pulling table from an Oracle database which has two columns: one is time and other is reportnumber. The requirement is, if the reportnumber increases above 45 continuously, then the alert needs to be triggered (Note: data is pulling from database every 5 min)

Below is the search I'm using currently, but it's not working as expected.

index=myindex sourectype=mydata | where reportnumber>45

Note: Alert frequency is every 15 min

Can any one help me on this ?

Thanks in advance

0 Karma

woodcock
Esteemed Legend

Use this search every 5 minutes as an Alert with timeframes from -15m@m to now:

index=myindex sourectype=mydata | timechart span=5m max(reportnumber) AS reportnumber | where reportnumber>45 | stats count AS HowManyOfLastThreeAreGreaterThan45

Then set the trigger to a Custom Condition of HowManyOfLastThreeAreGreaterThan45>=3

snehalk
Communicator

Hello woodcock,

Thanks for reply, when i configure above search and while creating alert am getting below error

Cannot parse alert condition. Unknown search command HowManyOfLastThreeAreGreaterThan45

Please let me know where am going wrong

0 Karma

woodcock
Esteemed Legend

Oops, change it to where HowManyOfLastThreeAreGreaterThan45>=3.

0 Karma

snehalk
Communicator

Hello woodcock,

Thanks for reply, will try this and test the alert.

0 Karma

skoelpin
SplunkTrust
SplunkTrust

Yes, put in your search, then go to 'Save As'>Alert

Give it a title, description, Alert type should be 'Scheduled', time range should be 'Run on a CRON Schedule'

Earliest = -15m@m
Latest = @m
Cron Expression = * * * * *
Trigger if number of results 'is greater than 45'

0 Karma
Get Updates on the Splunk Community!

Splunk + ThousandEyes: Correlate frontend, app, and network data to troubleshoot ...

 Are you tired of troubleshooting delays caused by siloed frontend, application, and network data? We've got a ...

Splunk Observability for AI

Don’t miss out on an exciting Tech Talk on Splunk Observability for AI!Discover how Splunk’s agentic AI ...

🔐 Trust at Every Hop: How mTLS in Splunk Enterprise 10.0 Makes Security Simpler

From Idea to Implementation: Why Splunk Built mTLS into Splunk Enterprise 10.0  mTLS wasn’t just a checkbox ...