Alerting

Alerts when indexing stops

gba8912
Explorer

Hello,

I am a noob at Splunk. I know there are a few posts on this already but I'm not able to find a solution for my specific problem. I want to make an alert for when indexing stops. I am using the following: | tstats latest(_time) as latest where index=* by host | where latest < relative_time(now(), "-1m")

Normally, I want the "-1m" to be "-1d" but i changed it to test the alert. I can see on the search that I have a result from an ip address that is not indexing events every minute so I know the search is working. When I save it as an alert however, I get no alerts. I have tried real-time and scheduled alerts to attempts a trigger.

Does anyone know why the alert doesnt work or if there is something off with the search I am trying to use?

 

Thanks!

Labels (3)
0 Karma
1 Solution

burwell
SplunkTrust
SplunkTrust

Hi. I don't use real time alerting. 

Try scheduled. For real-time I can't see the conditions when it would alert.

I will attach to typical kinds of alerting I do (oh only allowed on attachment per reply).. 

  1. Either when the result count is greater than 0 (see attachment)
  2. A field has some particular value (for that I do custom and then in the box I literally say 
search myfield > 3

 

 

View solution in original post

scelikok
SplunkTrust
SplunkTrust

Hi @gba8912,

tstats searches cannot run in realtime. You should change your alert schedule to "Scheduled".

 

If this reply helps you an upvote and "Accept as Solution" is appreciated.

burwell
SplunkTrust
SplunkTrust

Hi do you want to share exactly how you are setting up the alert condition and what are you using to alert? Can you check the _internal index for errors?

0 Karma

gba8912
Explorer

Hi Burwell,

 

Thanks for your reply. I have searched _internal and have 1,136,041 events. I don't really see any errors but maybe I don't know what to look for. Also, I am using the search I posted and saving it as an alert. I have attached a screenshot of the settings that I currently have. I have also tried using the scheduled time feature. Neither works. 

0 Karma

burwell
SplunkTrust
SplunkTrust

Hi. I don't use real time alerting. 

Try scheduled. For real-time I can't see the conditions when it would alert.

I will attach to typical kinds of alerting I do (oh only allowed on attachment per reply).. 

  1. Either when the result count is greater than 0 (see attachment)
  2. A field has some particular value (for that I do custom and then in the box I literally say 
search myfield > 3

 

 

gba8912
Explorer

Thanks! I was able to get the alerts to work with scheduled. 

0 Karma
Get Updates on the Splunk Community!

Splunk Observability for AI

Don’t miss out on an exciting Tech Talk on Splunk Observability for AI!Discover how Splunk’s agentic AI ...

Splunk Enterprise Security 8.x: The Essential Upgrade for Threat Detection, ...

Watch On Demand the Tech Talk on November 6 at 11AM PT, and empower your SOC to reach new heights! Duration: ...

Splunk Observability as Code: From Zero to Dashboard

For the details on what Self-Service Observability and Observability as Code is, we have some awesome content ...