Alerting

Alert not triggering based on tstats results

rhombus00
Engager

We've run into a few occassions where one of our network devices stops sending logs to Splunk. I have a tstats search based on the blog post here: https://www.splunk.com/en_us/blog/tips-and-tricks/how-to-determine-when-a-host-stops-sending-logs-to...

Here is the search expression I'm using:

| tstats latest(_time) as latest where index=index_name earliest=-1d by host
| eval recent = if(latest > relative_time(now(),"-15m"),1,0), realLatest = strftime(latest,"%c")
| where recent=0

My tstats search does return the hosts that have not sent any logs, but it never triggers when I use this search in an Alert. I noticed that the search only shows the hosts in the Statistics view and there are no Events. Is this why my event is not triggering?

I've found several other examples on this forum of people using tstats to detect when a host stops sending logs. Is there something special they are configuring in their Alert to trigger off of the statistics results?

Labels (1)
0 Karma
1 Solution

isoutamo
SplunkTrust
SplunkTrust

Hi

this should works. Are you sure that you have granted a role which allow you to run alerts?

r. Ismo

View solution in original post

0 Karma

isoutamo
SplunkTrust
SplunkTrust

Hi

this should works. Are you sure that you have granted a role which allow you to run alerts?

r. Ismo

0 Karma

rhombus00
Engager

You're right that it should work. Thanks for the sanity check.

I deleted the alert and started from scratch and it's working as expected now. I think I messed up the notification settings when I was originally tinkering with it. 

0 Karma

rhombus00
Engager

I don't know much about role/permissions but I have created dozens of other alerts. The only difference I can tell is all my other alerts use searches that return events when triggered. This search only returns results in the statistics section.

0 Karma

isoutamo
SplunkTrust
SplunkTrust

Ok, then you have that role.

How you have defined this alert? Can you give a screenshot about it?

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Agent Mode Engaged! Enchaining Agentic Operations with Splunk AI Assistant 2.0

    Are you ready to transform how your team handles complex data requests? We invite you to our upcoming ...

Announcing Modern Navigation: A New Era of Splunk User Experience

We are excited to introduce the Modern Navigation feature in the Splunk Platform, available to both cloud and ...

Modernize your Splunk Apps – Introducing Python 3.13 in Splunk

We are excited to announce that the upcoming releases of Splunk Enterprise 10.2.x and Splunk Cloud Platform ...