I have a setup that consists of a Search Head and 2 indexers in a cluster. I also use a self signed SSL certificate between the indexers and my universal forwarders.
For some reason, my UF is able to connect to the indexers, but no data is sent.
07-09-2016 00:21:15.670 +0000 INFO TcpOutputProc - Connected to idx=x.x.x.x:9997
On the Indexer is this warning:
WARN DateParserVerbose - Failed to parse timestamp. Defaulting to timestamp of previous event (Fri Jul 8 20:30:38 2016). Context: source::\
What else can I test to pinpoint my issue?
Sigh, I forgot to add the index my inputs.conf was going to, to the admin role "indexes searched by default". Sorry for wasting everyone's time! Rookie mistake.
You have a linebreaking/merging problem or a timestamping problem (the former often causes the latter). We need to see a few sample log events and your inputs.conf and props.conf files.
It's possible that the timestamp recognition is not working as expected and the events are indexed with an old timestamp.
Have you tried setting the time range to 'all time' and see if there are any events from this forwarder?
| metadata type=hosts index=* to see if the host is connected
Also have a look at http://wiki.splunk.com/Community:Troubleshooting_Monitor_Inputs