topic Few forwarders not sending data in Getting Data In

Few forwarders not sending data

Vetrikmr — Thu, 14 Dec 2017 18:41:14 GMT

Hey everyone, I have installed UF agents in 180 servers and i have seen the data coming to splunk yesterday. But now i have noticed three of them are not sending data, I mean i am seeing 177 hosts in splunk. So how can we find out what are those three UF's which is not sending data. I have configured all these using Deployment server. Same index and same sourcetype. And is there a way that we can get alerts when the forwarder stops sending data or gets any issue? I am using splunk 6.3. Thank you

Re: Few forwarders not sending data

somesoni2 — Thu, 14 Dec 2017 21:29:25 GMT

See this
http://docs.splunk.com/Documentation/Splunk/7.0.1/Troubleshooting/Cantfinddata

Re: Few forwarders not sending data

adonio — Thu, 14 Dec 2017 21:37:10 GMT

hello there,

when you say "not sending data" do you mean to the regular indexes or to _internal index?
try this to find out how many distinct forwarders are out there and who sends to _internal but does not send "data":

| tstats dc(host) as unique values(host) as hosts where index=_*
| mvexpand hosts
| appendcols [ | tstats values(host) as data_hosts where index=*]
| eval match = if(hosts=data_hosts,1,0)
| where match=0

if you have only 177 distinct forwarders, then you will probably will have to manuallt figure out where are the other 3.
if you have 180 sends to _internal then it means that those unique 3 either have wrong inputs on them or there is no data generated.

hope it helps

Re: Few forwarders not sending data

cboillot — Fri, 16 Feb 2018 16:11:06 GMT

I am not sure what this search is doing. I went line by line and was following up until I got to line 3. Line three adds a field with all the host, but just for the first entry, as least it does when I run them. From here, only the all 500+ host would still be listed at step five, except for the first one.

Is there something I am missing?

what I did get to work, or it seems like it works, is this: