Alerting

Alert condition considering previous itirations

praddasg
Path Finder

Hello All,

Using the below conditions (along with the required conditions) to configure alert

earliest=-5h
| head 100

The challenge facing is, sometimes observing it is considering the transactions which has previously already generated an alert
for example, an alert got generated at 12:00 PM and then again it is generating at 01:30 PM. This is happening because there were some failures between 11:00 AM and 11:30 AM and even if there is 1 or 2 failures around 1:25 PM, it is considering the failures from 11:00 AM to 11:30 AM.

The reason to consider earliest=-5h& head 100 is because the transactions for this application is very less. Thought of using suppression for a longer time (currently it is set to 1 hr) but that might give rise to situations were valid scenario might get miss.

Is there any other way (other than reducing the earliest time or head or increasing the suppression) to mitigate this?

Tags (1)
0 Karma

harshpatel
Contributor

Hi @praddasg,

Typically how this works is you have the alert search running in the same interval as what you define in its time range so it runs on exclusive events only (i.e. it doesn't overlap on to same events). So for example, if you set your search time-range to be 1 hour then you set up your alert search to run after 1-hour intervals.

Please elaborate if I am misunderstanding your question. I'd be happy to help.

Hope this helps.

praddasg
Path Finder

It does not seem to happening like that, it is considering the occurrences of the previously considered events as well

0 Karma

harshpatel
Contributor

Could you provide the exact stanza and config from your savedsearches.conf?

0 Karma

praddasg
Path Finder

Hi @harshpatel

I dont have access to the backend. I am only using the GUI. Is this something i can retrieve from the GUI?

Regards
Pradipto Dasgupta

0 Karma

harshpatel
Contributor

You can check what are the settings under Settings > Searches and Reports and click on edit on the search that you are having trouble with to see its details.

0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In September, the Splunk Threat Research Team had two releases of new security content via the Enterprise ...

New in Observability - Improvements to Custom Metrics SLOs, Log Observer Connect & ...

The latest enhancements to the Splunk observability portfolio deliver improved SLO management accuracy, better ...

Improve Data Pipelines Using Splunk Data Management

  Register Now   This Tech Talk will explore the pipeline management offerings Edge Processor and Ingest ...