Why is Splunk unable to index logs with very small sizes [in KB] but is able to parse other files from that directory?

[juhisaxena28]

Hi,

I have to monitor all files inside one directory. But the tiny sized files are not getting into Splunk while all other files are duly getting indexed. i used CRCSalt parameters and Below is my config settings for inputs file.

[monitor://L:\XYZ.2.0\XYZlogs\*]
disabled = false
index = app_XYZ
sourcetype = _json
crcSalt = Source in greater than and less than sign
initCrcLength = 256

Please tell us what am I missing out on.

Thanks

[awheatcr]

Did you ever resolve your problem? I am experiencing the same issue with very small files ( < 2KB ) that Splunk forwarder is missing/skipping. Sometimes, I can delete and re-create the log file and Splunk will pick it up but sometimes nothing will trigger the forwarder to send the file to the indexers.

[awheatcr]

DId you ever resolve this issue? I am experiencing issues where Splunk forwarder sometimes misses very small ~2KB files.

[woodcock]

You have the setting wrong. Use this exactly (do NOT change anything at all):

crcSalt=<SOURCE>

[juhisaxena28]

Yes its indeed the same settings.

crcSalt=SOURCE with angular brackets

[woodcock]

Do you LITERALLY have this:

crcSalt=<SOURCE>

Or have you substituted the word SOURCE for something else like this:

crcSalt=</your/path/file>

YOU MUST NOT DO THE LATTER! YOU MUST DO THE FORMER!

[juhisaxena28]

Yes i have done the former setting only.

[woodcock]

Then it should work. Deploy to forwarders and restart splunk.

[isachse]

Are the files smaller than the 256 bytes?

[juhisaxena28]

File size is like 1-5KBs.

[juhisaxena28]

Also i just discovered that few of the data is going into "lastchanceindex". Why is that the case.

[Vijeta]

make sure the path is correct, try giving complete file name.

[juhisaxena28]

Yes path is accurate given other large files are duly getting indexed in splunk.