Alerting

How do you auto-clear alerts for log sources that begin sending logs again after x-seconds?

cemiam
Path Finder

Hi,

I have an alert for log sources that stopped sending logs for a while. Alert string is like: | metadata type=sources | eval age=now()-lastTime | search age>600

I am planning to clear alerts for the log sources that started to send logs after 300 seconds. Is there any way to do that? I thought log sources might be written into a reference set with the inputlookup command, but I am not able to find how to clear values that start sending logs.

Best Regards,
Cem

1 Solution

jkat54
SplunkTrust
SplunkTrust

I have updated the Splunk-TA_webtools app on splunkbase to support this scenario:

https://splunkbase.splunk.com/app/3420

Deleting 10 fired alerts for search named "Test Alert":
| rest /servicesNS/admin/search/alerts/fired_alerts/Test%20Alert
| fields title
| head 10
| map search="|curl method=delete ssl=true uri="localhost:8089/servicesNS/admin/search/alerts/fired_alerts/$title$" user=admin pass=changeme | table *"

View solution in original post

jkat54
SplunkTrust
SplunkTrust

I have updated the Splunk-TA_webtools app on splunkbase to support this scenario:

https://splunkbase.splunk.com/app/3420

Deleting 10 fired alerts for search named "Test Alert":
| rest /servicesNS/admin/search/alerts/fired_alerts/Test%20Alert
| fields title
| head 10
| map search="|curl method=delete ssl=true uri="localhost:8089/servicesNS/admin/search/alerts/fired_alerts/$title$" user=admin pass=changeme | table *"

jkat54
SplunkTrust
SplunkTrust

@cemiam this is working for me, please test on your end after downloading and installing the last Splunk-TA_webtool (v1.00).

cemiam
Path Finder

Hi jkat54,

It worked like a charm. Thank you for your efforts. It seems this will resolve our problem.

Best Regards,

jkat54
SplunkTrust
SplunkTrust

That's great! Happy to help.

0 Karma

jkat54
SplunkTrust
SplunkTrust

@cemiam if you don't mind, can you please take a moment to rate the app on splunkbase? Thanks again!

0 Karma

cemiam
Path Finder

Sure. I have rated your app. Thanks again for your assitance.

niketn
Legend

By clearing the alert do you mean you want to clear already triggered alert from Triggered Alert History or do you want to change your alert condition from sources not feeding data in last 10 min to last 5 min (300 sec)?

| metadata type=sources 
| eval age=now()-lastTime 
| search age>300
____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

cemiam
Path Finder

I want to clear alert if it send logs in last 5 mins. Is there any method to clear alerts? Actually it don't have to be an alert. We can just save it as a report. The important part is clearing the values after 5 minutes if it started to send logs.

0 Karma

niketn
Legend

@cemiam, You can create a dashboard with source details which refreshes every minutes.

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

cemiam
Path Finder

Hi,

This time I'll not see if problem occurs and resolves in a few minutes. It will constantly refresh the report and delete every source which starts to send log. We need to delete them only after they send at most in 5 minutes. Do you know any way to do this?

Best Regards,

0 Karma
Get Updates on the Splunk Community!

Financial Services Industry Use Cases, ITSI Best Practices, and More New Articles ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Splunk Federated Analytics for Amazon Security Lake

Thursday, November 21, 2024  |  11AM PT / 2PM ET Register Now Join our session to see the technical ...

Splunk With AppDynamics - Meet the New IT (And Engineering) Couple

Wednesday, November 20, 2024  |  10AM PT / 1PM ET Register Now Join us in this session to learn all about ...