Monitoring Splunk

Splunk Log ingestion issues

Prakash493
Communicator

Currently i am facing a issue , i am monitoring a directory that has over 14000 files i am getting few files are ingesting to splunk and few files are not ingesting , currently i am using these two stanzas in my input.conf file disabled = false
recursive = true , anyone has face similar issue ?

Tags (1)
0 Karma
1 Solution

koshyk
Super Champion

This could be related to multiple issues
1. crcSalt => By default, the Splunk only performs CRCs against the first few lines of a file. so if the lines are exactly same on these files for the default settings, it may NOT index
2. initCrcLength => default is 256 bytes ; How much of a file, in bytes, that the input reads before trying to
identify whether it is a file that has already been seen. You might want to adjust this if you have many files with common headers
3. Do a btool of inputs to see if your settings are overriden in some other app

you need to give the exact samples/btool output in the question for us to help you further

Good link to read about => https://docs.splunk.com/Documentation/Splunk/7.2.6/Data/Monitorfilesanddirectorieswithinputs.conf
and of course inputs.conf specification

View solution in original post

0 Karma

koshyk
Super Champion

This could be related to multiple issues
1. crcSalt => By default, the Splunk only performs CRCs against the first few lines of a file. so if the lines are exactly same on these files for the default settings, it may NOT index
2. initCrcLength => default is 256 bytes ; How much of a file, in bytes, that the input reads before trying to
identify whether it is a file that has already been seen. You might want to adjust this if you have many files with common headers
3. Do a btool of inputs to see if your settings are overriden in some other app

you need to give the exact samples/btool output in the question for us to help you further

Good link to read about => https://docs.splunk.com/Documentation/Splunk/7.2.6/Data/Monitorfilesanddirectorieswithinputs.conf
and of course inputs.conf specification

0 Karma

Prakash493
Communicator

so do i need to add crcSalt= or just crcSalt =>

0 Karma
Get Updates on the Splunk Community!

Webinar Recap | Revolutionizing IT Operations: The Transformative Power of AI and ML ...

The Transformative Power of AI and ML in Enhancing Observability   In the realm of IT operations, the ...

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...