topic Re: How do I alert when a host stops sending data? in Splunk Search

How do I alert when a host stops sending data?

matt — Wed, 02 Jun 2010 05:08:50 GMT

What's the best way to create a search to identify which hosts have not sent a syslog message to Splunk in the last 2 days?

Re: How do I alert when a host stops sending data?

Lowell — Wed, 02 Jun 2010 05:25:51 GMT

Are you talking specifically sourcetype=syslog or just any events from a host? It's easy to do any events from a host with something like this:

| metadata index=main type=hosts | eval age = now()-lastTime | where age > (2*86400) | sort age d | convert ctime(lastTime) | fields age,host,lastTime

Does that work for you?

Re: How do I alert when a host stops sending data?

ram_malhotra — Fri, 18 Jun 2010 01:33:21 GMT

I tried this search an got 0 search results

Re: How do I alert when a host stops sending data?

CerielTjuh — Fri, 18 Jun 2010 13:37:17 GMT

this could mean that you don't have any "lost" hosts

Re: How do I alert when a host stops sending data?

ckurtz — Tue, 12 Aug 2014 20:33:18 GMT

Or that you don't use the main index. Try doing

This will search all non-internal indexes and display any hosts that haven't reported in 2 seconds.

Re: How do I alert when a host stops sending data?

r34220 — Tue, 13 Dec 2016 19:05:22 GMT

I am getting the point where I feel ITSI may not be the right choice for service monitoring for us. How can you rely on ITSI if it can't easily detect if a KPI for a Entity is not getting data in a ITSI Service? What I am finding is if any query returns no results ITSI keeps the last KPI value for an Entity it received. Since Splunk ITSI is used in a large enterprise by several business units, it is imperative that each BU can tell if a Entities not reporing in within a ITSI Service.

In my example, i can tell by the Aggregate Calculation that a host is not reporting in but how can I easily tell which Entity? Only the Entities that have previously stop sending data will show in the Entities section but never drop off once it starts getting data again.

I am using the query

| metadata index=* type=hosts | eval age = now()-lastTime | where age > 300 | sort age d | convert ctime(lastTime) | fields age,host,lastTime

Does anyone have a sure way of knowing which Entity has stopped sending data within a ITSI Service?

Re: How do I alert when a host stops sending data?

mbain606 — Tue, 14 Mar 2017 15:49:37 GMT

I downvoted this post because this is not suitable for the original question and should not be ranked as an answer because:
1. it is a question about another service
2. the person posted it as an answer when it is really just another question.

Re: How do I alert when a host stops sending data?

zward — Thu, 07 Dec 2017 15:53:33 GMT

I downvoted this post because not relevant to the question, and it is another question -- not an answer or solution to the original question.

Re: How do I alert when a host stops sending data?

bestSplunker — Wed, 30 May 2018 06:53:11 GMT

@matt hey,guys
you can get sourcetype which is stop

|tstats count as countAtToday latest(_time) as lastTime where index!="*_" by host sourcetype|eval age=now()-lastTime|sort age d|fieldformat lastTime=strftime(lastTime,"%Y/%m/%d %H:%M:%S")|eval age=round((age/60/60),1)|search age>=48|eval age=age."hour"

Re: How do I alert when a host stops sending data?

raja8220 — Mon, 15 Apr 2019 12:54:53 GMT

if do search am getting 5 host name and time if I create alert for it its not triggering ??

Re: How do I alert when a host stops sending data?

raja8220 — Mon, 15 Apr 2019 12:55:32 GMT

if do search am getting 5 host name in statistics and time if I create alert for it its not triggering ??