Getting Data In
Highlighted

Splunk indexing rate high with no new files in directory monitor

Explorer

We are "monitoring" a directory on a Splunk indexer host. This is the only input on this particular indexer. The indexing rate has remained constant at 5,000KB/s although no new files have been added to the directory for a few days. How is this possible? Could Splunk be re-indexing the files in this directory? The files have not changed in any way since they were copied over.

0 Karma
Highlighted

Re: Splunk indexing rate high with no new files in directory monitor

Path Finder

call support. they'll tell you they'll call you back, then not call you back...

0 Karma
Highlighted

Re: Splunk indexing rate high with no new files in directory monitor

Champion

Ok, Not sure whats happening. So do you have have schedule task or cron job copy the directories/files over? Are you using CRCsalt? Install SOS (Splunk on Splunk) App or search the _internal index for the source which may give you some ideas.

Considerchanging your File checksum configuration to [endpoint_md5|entire_md5|modtime] in your props.conf.

If no luck up your logging channels on your indexer (I am not familar with all the logging channels):

  • FileInputTracker
  • FileTracker
  • recordFileManager
  • WatchedFile

Also keep in mind that Splunk indexes its own logs as well as those from your Forwarders which is stored in the _internal index. _internal does not count against your license.

Hope this helps or gets you started.

Speak Up for Splunk Careers!

We want to better understand the impact Splunk experience and expertise has has on individuals' careers, and help highlight the growing demand for Splunk skills.