Getting Data In

Universal Forwarder not sending data to indexer after successful connection

RecoMark0
Path Finder

Hello,
I have a setup that consists of a Search Head and 2 indexers in a cluster. I also use a self signed SSL certificate between the indexers and my universal forwarders.

For some reason, my UF is able to connect to the indexers, but no data is sent.
07-09-2016 00:21:15.670 +0000 INFO TcpOutputProc - Connected to idx=x.x.x.x:9997

  1. My test logs directly on the indexers were sent to the search head without issue.
  2. running splunk list monitor on the UF lists all the logs I want to monitor
  3. No errors in splunkd.log on the UF that I can see, just the usual warnings I get in my duplicate setup in another enviornment that IS working.
  4. No errors in metrics.log on the UF either.
  5. On the Indexer is this warning:

    WARN DateParserVerbose - Failed to parse timestamp. Defaulting to timestamp of previous event (Fri Jul 8 20:30:38 2016). Context: source::\\s$\Logs\service.log|host::|Service Logs|174315

What else can I test to pinpoint my issue?

0 Karma
1 Solution

ddrillic
Ultra Champion

The "official" documentation to debug such a case at I can't find my data!

View solution in original post

ddrillic
Ultra Champion

The "official" documentation to debug such a case at I can't find my data!

RecoMark0
Path Finder

Sigh, I forgot to add the index my inputs.conf was going to, to the admin role "indexes searched by default". Sorry for wasting everyone's time! Rookie mistake.

0 Karma

ddrillic
Ultra Champion

It's all good - we all make all sorts of mistakes...

woodcock
Esteemed Legend

You have a linebreaking/merging problem or a timestamping problem (the former often causes the latter). We need to see a few sample log events and your inputs.conf and props.conf files.

renjith_nair
SplunkTrust
SplunkTrust

It's possible that the timestamp recognition is not working as expected and the events are indexed with an old timestamp.
Have you tried setting the time range to 'all time' and see if there are any events from this forwarder?

Try | metadata type=hosts index=* to see if the host is connected

Also have a look at http://wiki.splunk.com/Community:Troubleshooting_Monitor_Inputs

Happy Splunking!
Get Updates on the Splunk Community!

NEW! Log Views in Splunk Observability Dashboards Gives Context From a Single Page

Today, Splunk Observability releases log views, a new feature for users to add their logs data from Splunk Log ...

Last Chance to Submit Your Paper For BSides Splunk - Deadline is August 12th!

Hello everyone! Don't wait to submit - The deadline is August 12th! We have truly missed the community so ...

Ready, Set, SOAR: How Utility Apps Can Up Level Your Playbooks!

 WATCH NOW Powering your capabilities has never been so easy with ready-made Splunk® SOAR Utility Apps. Parse ...