How to edit my search to find hosts and sources that are not sending logs to Splunk?

[syed_star357]

Hi Team,

How to search which are the hosts and Sources not sending logs to Splunk? The below metadata search shows only host. How can I include source in results?

I need columns for source, Host, lasttime, and duration.

|metadata index!=network* index=win* index=lin* type=hosts  | table host sourcetype lastTime | stats max(lastTime) as lastTime by host | eval diff = now()-lastTime | where diff > 3600|sort - diff | eval lastTime=strftime(lastTime,"%Y-%m-%d %H:%M:%S") |eval Duration=tostring(diff,"duration")|fields - diff

Regards,
Syed

[renjith_nair · Accepted solution]

Try this

|tstats latest(_time) as lastTime where (index=win* OR index=lin*) by host,source
|eval diff = now()-lastTime | where diff > 3600
|sort - diff | eval lastTime=strftime(lastTime,"%Y-%m-%d %H:%M:%S") 
|eval Duration=tostring(diff,"duration")|fields - diff

You need to select timerange according to your requirements

[inventsekar]

Please check this.. just did a reordering. You will need to add that table command in the last..

 |metadata index!=network* index=win* index=lin* type=hosts | stats max(lastTime) as lastTime by host | eval diff = now()-lastTime | where diff > 3600|sort - diff | eval lastTime=strftime(lastTime,"%Y-%m-%d %H:%M:%S") |eval Duration=tostring(diff,"duration") | table host sourcetype lastTime

[syed_star357]

metadata take one values only host or either sourcetype, if i give below search will get only blank data in sourcetype and lasttime fields.

|metadata index!=network* index=win* index=lin* type=hosts | stats max(lastTime) as lastTime by host | eval diff = now()-lastTime | where diff > 3600|sort - diff | eval lastTime=strftime(lastTime,"%Y-%m-%d %H:%M:%S") |eval Duration=tostring(diff,"duration") | table host sourcetype lastTime

[inventsekar]

Hi Syed, can you update us if you are able to find the devices which are not sending logs..
if you have missed, as you are a new member, can you please accept this as an answer

[inventsekar]

You can run the following search to detect forwarders that have been up in the last 24 hours but not in the last 2 minutes. It uses the forwarder heartbeat, which is a feature of Splunk versions 3.2 and later.

index=_internal sourcetype="fwd-hb" starthoursago=24 | dedup host | eval age = strftime("%s","now") - _time | search age > 120 age < 86000

You can set this search up as an alert every several minutes so that Splunk will let you know if any of your active forwarders have not responded in the last 2 minutes.

If you're running a version of Splunk that is later than 3.3', the heartbeat message is not longer sent. Use the following search instead:

index=_internal "group=tcpin_connections" | stats max(_time) as latest by sourceHost | eventstats max(latest) as latest_all | eval lag = latest_all - latest | where lag > 120 | fields sourceHost lag

[inventsekar]

I need to column of source,Host,lasttime and duration. // Finding source and host together may not be a good method I think. For example, if a host has not sent an event means, all sources/sourcetype s from that host also has not an event.
A source/sourcetype which is sending events properly from one host may not be sending any events from another host. So finding source/sourcetype may not be useful I think.
Or, do you have other reasons for looking source/sourcetype?