Dashboards & Visualizations

Why Splunk doesn't index all the lines in a monitored file?

andrei_radu
New Member

Hello,

I use Splunk Universal Forwarder 6.3.4 on a centralized syslog collector and it's set to monitor a folder populated with symbolic links. (inputs.conf content below).

[monitor:///service/rsyslog/splunk/critical_systems/]
followSymlink = true
host_segment=5
sourcetype = critical_unix
index = critical_systems

Everything went good until i noticed that not all the lines inside the monitored files are being indexed. I'm not filtering anything with props.conf and transforms.conf. There's no pattern for the lines that are skipped, everything is random.

Any idea why this is happening and how to fix it?

Thanks!

0 Karma

andrei_radu
New Member

I found my problem. There were 2 different apps monitoring the same location, sending data to different indexes. So logs went randomly into only one index. Disabled one of the apps and now everything looks fine.

0 Karma

ddrillic
Ultra Champion

When we index, we make an assumption that the Splunk time stamp process works well.

We see here an example where this process didn't work as expected - Can't index new data..? :S

It says -

alt text

0 Karma

lguinn2
Legend

Do you see any messages in $SPLUNK_HOME/var/log/splunk/splunkd.log? In particular, I would look at this log file on the forwarder

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...