I use Splunk Universal Forwarder 6.3.4 on a centralized syslog collector and it's set to monitor a folder populated with symbolic links. (inputs.conf content below).
followSymlink = true
sourcetype = criticalunix
index = criticalsystems
Everything went good until i noticed that not all the lines inside the monitored files are being indexed. I'm not filtering anything with props.conf and transforms.conf. There's no pattern for the lines that are skipped, everything is random.
Any idea why this is happening and how to fix it?
Do you see any messages in $SPLUNK_HOME/var/log/splunk/splunkd.log? In particular, I would look at this log file on the forwarder
I found my problem. There were 2 different apps monitoring the same location, sending data to different indexes. So logs went randomly into only one index. Disabled one of the apps and now everything looks fine.