Getting Data In

How to troubleshoot why a file in monitored folder is not being indexed in Splunk Light Free 6.3.0?

gbeddow
Explorer

Hi,

I’ve been using Splunk Light Free Version 6.3.0 for about a month on Mac OS X, and it’s been working well, monitoring 16 modestly-sized log files updated once per day, taken from 3 different websites.

Here’s the relevant part of inputs.conf:

[monitor:///Users/user/backup-log]
disabled = false
host = user.com
blacklist = \/\.[^\/]*$
sourcetype=access_combined

Recently I noticed a newly-added log file (#17) in a monitored folder wasn’t being indexed. At first thinking there may be a limit on the number of sources in the free version, I removed a couple of older log files from the monitored folder, then cleaned out all the index data and reindexed everything:

$ /Applications/Splunk/bin/splunk stop
$ /Applications/Splunk/bin/splunk clean eventdata
$ /Applications/Splunk/bin/splunk start

But it simply indexed 14 files instead of 16, still omitting the newly-added log file.

So then I tried renaming the newly-added log file and reindexing everything. Interestingly, the renamed file got indexed, but some other file was no longer indexed (and it was being indexed before).

Any ideas out there on how to fix this?

Thanks,
Greg

1 Solution

MuS
SplunkTrust
SplunkTrust

Hi gbeddow,

check splunkd.log or index=_internal for any message related to the missing file. Do some reading here http://docs.splunk.com/Documentation/Splunk/6.3.0/Data/HowLogFileRotationIsHandled and learn how Splunk uses the file header to do CRC; maybe you have to adjust initCrcLength in inputs.conf http://docs.splunk.com/Documentation/Splunk/6.3.0/admin/Inputsconf

Also

Hope this helps ...

cheers, MuS

View solution in original post

MuS
SplunkTrust
SplunkTrust

Hi gbeddow,

check splunkd.log or index=_internal for any message related to the missing file. Do some reading here http://docs.splunk.com/Documentation/Splunk/6.3.0/Data/HowLogFileRotationIsHandled and learn how Splunk uses the file header to do CRC; maybe you have to adjust initCrcLength in inputs.conf http://docs.splunk.com/Documentation/Splunk/6.3.0/admin/Inputsconf

Also

Hope this helps ...

cheers, MuS

View solution in original post

gbeddow
Explorer

Hi MuS,

I looked at splunkd.log and found this from when Splunk first saw the newly-added log file in question:

11-01-2015 06:56:44.400 -0800 ERROR TailReader - File will not be read, is too small to match seekptr checksum (file=<some-file-name>).  Last time we saw this initcrc, filename was different.  You may wish to use larger initCrcLen for this sourcetype, or a CRC salt on this source.  Consult the documentation or file a support case online at http://www.splunk.com/page/submit_issue for more info.

Then I looked at these:

http://docs.splunk.com/Documentation/Splunk/6.3.0/Data/HowLogFileRotationIsHandled
http://docs.splunk.com/Documentation/Splunk/6.3.0/Data/Monitorfilesanddirectorieswithinputs.conf

So I added this to each stanza in inputs.conf:

crcSalt = <SOURCE>

then reindexed everything and that took care of the problem.

Thanks for your help!

--Greg

0 Karma

BansodeSantosh
Explorer

crcSalt =
This worked for me....Great!!

0 Karma

MuS
SplunkTrust
SplunkTrust

Watch out when using crcSalt = this can give you double indexed events! If you encounter double indexed events, change the initCrcLen and discard crcSalt = 😉

0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.