topic Re: Why is Splunk universal forwarder not indexing data from all log files? in Getting Data In

Why is Splunk universal forwarder not indexing data from all log files?

AzmathShaik — Tue, 07 Feb 2017 20:27:33 GMT

Hello

I am running Splunk as not root user. my Splunk universal forwarder is not indexing data from all files.

when i run Splunk list monitor, it is listing all my files that i have mentioned in input stanza, but it is not indexing those files. i have checked privileges for those files and as a non root user i can cat those files.

can anyone help me??

Re: Why is Splunk universal forwarder not indexing data from all log files?

woodcock — Tue, 07 Feb 2017 20:37:22 GMT

What do you get with this search:

index=_* host=<YourHost>

If nothing, then the problem is that your host is not sending to your indexers.

Re: Why is Splunk universal forwarder not indexing data from all log files?

s2_splunk — Tue, 07 Feb 2017 20:40:35 GMT

Yup, second stop would be to check /opt/splunkforwarder/var/log/splunk/splunkd.log directly on the forwarder host for any hints as to what may be happening.

Re: Why is Splunk universal forwarder not indexing data from all log files?

AzmathShaik — Tue, 07 Feb 2017 20:41:35 GMT

it is giving me events with _internal index.

Re: Why is Splunk universal forwarder not indexing data from all log files?

AzmathShaik — Tue, 07 Feb 2017 20:43:18 GMT

i have checked splunkd.log there are no errors

Re: Why is Splunk universal forwarder not indexing data from all log files?

woodcock — Tue, 07 Feb 2017 21:21:30 GMT

Check to make sure that the clock on the forwarder is correct and the TZ is accounted for in the TZ setting in props.conf. It may be that the events are being "sent to the future". To check this, search for "All Time".

Also login to the server as the user that is running splunk and verify that you can read the file contents manually. If you have a permission problem manually, then this is the problem for Splunk, too.

Re: Why is Splunk universal forwarder not indexing data from all log files?

s2_splunk — Tue, 29 Sep 2020 12:47:55 GMT

Next thing to check would be whether the index you have configured for each input on the forwarder are defined on the indexing tier. Otherwise you will see error messages in _internal that look like this: "received event for unconfigured/disabled/deleted index='[index_name]' with source='[source]' host='[hostname]' sourcetype='[sourcetype]' (n missing total)"

You can also run this search to find out whether the indexer is receiving any data from the forwarder:
index=_internal sourcetype=splunkd group=per_host_thruput series=your_forwarder_host_here | timechart sum(kb) as totalkb by series limit=0