Splunk Search

How to trigger an alert if 3 consecutive search results reach a certain threshold?

snehalk
Communicator

Hello All,

I have one requirement where an alert needs to be triggered after three continuous search results reach a certain threshold.

Eg: am pulling table from an Oracle database which has two columns: one is time and other is reportnumber. The requirement is, if the reportnumber increases above 45 continuously, then the alert needs to be triggered (Note: data is pulling from database every 5 min)

Below is the search I'm using currently, but it's not working as expected.

index=myindex sourectype=mydata | where reportnumber>45

Note: Alert frequency is every 15 min

Can any one help me on this ?

Thanks in advance

0 Karma

woodcock
Esteemed Legend

Use this search every 5 minutes as an Alert with timeframes from -15m@m to now:

index=myindex sourectype=mydata | timechart span=5m max(reportnumber) AS reportnumber | where reportnumber>45 | stats count AS HowManyOfLastThreeAreGreaterThan45

Then set the trigger to a Custom Condition of HowManyOfLastThreeAreGreaterThan45>=3

snehalk
Communicator

Hello woodcock,

Thanks for reply, when i configure above search and while creating alert am getting below error

Cannot parse alert condition. Unknown search command HowManyOfLastThreeAreGreaterThan45

Please let me know where am going wrong

0 Karma

woodcock
Esteemed Legend

Oops, change it to where HowManyOfLastThreeAreGreaterThan45>=3.

0 Karma

snehalk
Communicator

Hello woodcock,

Thanks for reply, will try this and test the alert.

0 Karma

skoelpin
SplunkTrust
SplunkTrust

Yes, put in your search, then go to 'Save As'>Alert

Give it a title, description, Alert type should be 'Scheduled', time range should be 'Run on a CRON Schedule'

Earliest = -15m@m
Latest = @m
Cron Expression = * * * * *
Trigger if number of results 'is greater than 45'

0 Karma
Get Updates on the Splunk Community!

Updated Team Landing Page in Splunk Observability

We’re making some changes to the team landing page in Splunk Observability, based on your feedback. The ...

New! Splunk Observability Search Enhancements for Splunk APM Services/Traces and ...

Regardless of where you are in Splunk Observability, you can search for relevant APM targets including service ...

Webinar Recap | Revolutionizing IT Operations: The Transformative Power of AI and ML ...

The Transformative Power of AI and ML in Enhancing Observability   In the realm of IT operations, the ...