Reporting

Trigger alert when saved search has error

BernardEAI
Communicator

We have a number of saved searches (configured as alerts) that make use of custom search commands we wrote. It can happen that those custom commands fail to execute. This would cause the saved search to fail with an error message, which can be seen if you would run the search in the Splunk UI.

We would like alerts to be triggered if this happens to the saved search. In the Alert configuration, this type of trigger is not an option. 

Is there any way we can get an alert triggered when one of our saved searches fails with an error?

PS: we are running our app on a multi-tenant platform, so we do not have access to the internal logs, thus we cannot run a search like:

index=_internal sourcetype=scheduler status!=success 
| table _time search_type status user app savedsearch_name 

 

Labels (2)
0 Karma
1 Solution

BernardEAI
Communicator

@venkatasri thanks for the suggestion. 

I have tried getting the status of the saved search using an API call. I have run this API:

/servicesNS/admin/<app_name>/saved/searches/<save_search_name>/history

This returns a lot of the info on the search, but not the status!?!? I would have expected this would be in the data. Any idea if there is another endpoint that I can use that would have the status info?

UPDATE:

I managed to find an API call that would report on failed scheduled searches:

| rest /servicesNS/-/<app_name>/search/jobs | search isFailed=1
| table label, published, messages.fatal

This will work on our multi-tenant platform.

View solution in original post

0 Karma

venkatasri
SplunkTrust
SplunkTrust

Hi @BernardEAI 

Did you try Rest API's provided by Splunk if you got access to API's?

0 Karma

BernardEAI
Communicator

@venkatasri thanks for the suggestion. 

I have tried getting the status of the saved search using an API call. I have run this API:

/servicesNS/admin/<app_name>/saved/searches/<save_search_name>/history

This returns a lot of the info on the search, but not the status!?!? I would have expected this would be in the data. Any idea if there is another endpoint that I can use that would have the status info?

UPDATE:

I managed to find an API call that would report on failed scheduled searches:

| rest /servicesNS/-/<app_name>/search/jobs | search isFailed=1
| table label, published, messages.fatal

This will work on our multi-tenant platform.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...