Hi,
I’ve been using Splunk Light Free Version 6.3.0 for about a month on Mac OS X, and it’s been working well, monitoring 16 modestly-sized log files updated once per day, taken from 3 different websites.
Here’s the relevant part of inputs.conf:
[monitor:///Users/user/backup-log]
disabled = false
host = user.com
blacklist = \/\.[^\/]*$
sourcetype=access_combined
Recently I noticed a newly-added log file (#17) in a monitored folder wasn’t being indexed. At first thinking there may be a limit on the number of sources in the free version, I removed a couple of older log files from the monitored folder, then cleaned out all the index data and reindexed everything:
$ /Applications/Splunk/bin/splunk stop
$ /Applications/Splunk/bin/splunk clean eventdata
$ /Applications/Splunk/bin/splunk start
But it simply indexed 14 files instead of 16, still omitting the newly-added log file.
So then I tried renaming the newly-added log file and reindexing everything. Interestingly, the renamed file got indexed, but some other file was no longer indexed (and it was being indexed before).
Any ideas out there on how to fix this?
Thanks,
Greg
Hi gbeddow,
check splunkd.log
or index=_internal
for any message related to the missing file. Do some reading here http://docs.splunk.com/Documentation/Splunk/6.3.0/Data/HowLogFileRotationIsHandled and learn how Splunk uses the file header to do CRC; maybe you have to adjust initCrcLength
in inputs.conf
http://docs.splunk.com/Documentation/Splunk/6.3.0/admin/Inputsconf
Also
blacklist
is not matching the missing fileTailingProcess
Hope this helps ...
cheers, MuS
Hi gbeddow,
check splunkd.log
or index=_internal
for any message related to the missing file. Do some reading here http://docs.splunk.com/Documentation/Splunk/6.3.0/Data/HowLogFileRotationIsHandled and learn how Splunk uses the file header to do CRC; maybe you have to adjust initCrcLength
in inputs.conf
http://docs.splunk.com/Documentation/Splunk/6.3.0/admin/Inputsconf
Also
blacklist
is not matching the missing fileTailingProcess
Hope this helps ...
cheers, MuS
Hi MuS,
I looked at splunkd.log and found this from when Splunk first saw the newly-added log file in question:
11-01-2015 06:56:44.400 -0800 ERROR TailReader - File will not be read, is too small to match seekptr checksum (file=<some-file-name>). Last time we saw this initcrc, filename was different. You may wish to use larger initCrcLen for this sourcetype, or a CRC salt on this source. Consult the documentation or file a support case online at http://www.splunk.com/page/submit_issue for more info.
Then I looked at these:
http://docs.splunk.com/Documentation/Splunk/6.3.0/Data/HowLogFileRotationIsHandled
http://docs.splunk.com/Documentation/Splunk/6.3.0/Data/Monitorfilesanddirectorieswithinputs.conf
So I added this to each stanza in inputs.conf:
crcSalt = <SOURCE>
then reindexed everything and that took care of the problem.
Thanks for your help!
--Greg
crcSalt =
This worked for me....Great!!
Watch out when using crcSalt =
this can give you double indexed events! If you encounter double indexed events, change the initCrcLen
and discard crcSalt =
😉