Alerting

Alerts when indexing stops

gba8912
Explorer

Hello,

I am a noob at Splunk. I know there are a few posts on this already but I'm not able to find a solution for my specific problem. I want to make an alert for when indexing stops. I am using the following: | tstats latest(_time) as latest where index=* by host | where latest < relative_time(now(), "-1m")

Normally, I want the "-1m" to be "-1d" but i changed it to test the alert. I can see on the search that I have a result from an ip address that is not indexing events every minute so I know the search is working. When I save it as an alert however, I get no alerts. I have tried real-time and scheduled alerts to attempts a trigger.

Does anyone know why the alert doesnt work or if there is something off with the search I am trying to use?

 

Thanks!

Labels (3)
0 Karma
1 Solution

burwell
SplunkTrust
SplunkTrust

Hi. I don't use real time alerting. 

Try scheduled. For real-time I can't see the conditions when it would alert.

I will attach to typical kinds of alerting I do (oh only allowed on attachment per reply).. 

  1. Either when the result count is greater than 0 (see attachment)
  2. A field has some particular value (for that I do custom and then in the box I literally say 
search myfield > 3

 

 

View solution in original post

scelikok
SplunkTrust
SplunkTrust

Hi @gba8912,

tstats searches cannot run in realtime. You should change your alert schedule to "Scheduled".

 

If this reply helps you an upvote and "Accept as Solution" is appreciated.

burwell
SplunkTrust
SplunkTrust

Hi do you want to share exactly how you are setting up the alert condition and what are you using to alert? Can you check the _internal index for errors?

0 Karma

gba8912
Explorer

Hi Burwell,

 

Thanks for your reply. I have searched _internal and have 1,136,041 events. I don't really see any errors but maybe I don't know what to look for. Also, I am using the search I posted and saving it as an alert. I have attached a screenshot of the settings that I currently have. I have also tried using the scheduled time feature. Neither works. 

0 Karma

burwell
SplunkTrust
SplunkTrust

Hi. I don't use real time alerting. 

Try scheduled. For real-time I can't see the conditions when it would alert.

I will attach to typical kinds of alerting I do (oh only allowed on attachment per reply).. 

  1. Either when the result count is greater than 0 (see attachment)
  2. A field has some particular value (for that I do custom and then in the box I literally say 
search myfield > 3

 

 

gba8912
Explorer

Thanks! I was able to get the alerts to work with scheduled. 

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

[Puzzles] Solve, Learn, Repeat: Character substitutions with Regular Expressions

This challenge was first posted on Slack #puzzles channelFor BORE at .conf23, we had a puzzle question which ...

Splunk Community Badges!

  Hey everyone! Ready to earn some serious bragging rights in the community? Along with our existing badges ...

[Puzzles] Solve, Learn, Repeat: Matching cron expressions

This puzzle (first published here) is based on matching timestamps to cron expressions.All the timestamps ...