Monitoring Splunk

Splunk Log ingestion issues

Prakash493
Communicator

Currently i am facing a issue , i am monitoring a directory that has over 14000 files i am getting few files are ingesting to splunk and few files are not ingesting , currently i am using these two stanzas in my input.conf file disabled = false
recursive = true , anyone has face similar issue ?

Tags (1)
0 Karma
1 Solution

koshyk
Super Champion

This could be related to multiple issues
1. crcSalt => By default, the Splunk only performs CRCs against the first few lines of a file. so if the lines are exactly same on these files for the default settings, it may NOT index
2. initCrcLength => default is 256 bytes ; How much of a file, in bytes, that the input reads before trying to
identify whether it is a file that has already been seen. You might want to adjust this if you have many files with common headers
3. Do a btool of inputs to see if your settings are overriden in some other app

you need to give the exact samples/btool output in the question for us to help you further

Good link to read about => https://docs.splunk.com/Documentation/Splunk/7.2.6/Data/Monitorfilesanddirectorieswithinputs.conf
and of course inputs.conf specification

View solution in original post

0 Karma

koshyk
Super Champion

This could be related to multiple issues
1. crcSalt => By default, the Splunk only performs CRCs against the first few lines of a file. so if the lines are exactly same on these files for the default settings, it may NOT index
2. initCrcLength => default is 256 bytes ; How much of a file, in bytes, that the input reads before trying to
identify whether it is a file that has already been seen. You might want to adjust this if you have many files with common headers
3. Do a btool of inputs to see if your settings are overriden in some other app

you need to give the exact samples/btool output in the question for us to help you further

Good link to read about => https://docs.splunk.com/Documentation/Splunk/7.2.6/Data/Monitorfilesanddirectorieswithinputs.conf
and of course inputs.conf specification

0 Karma

Prakash493
Communicator

so do i need to add crcSalt= or just crcSalt =>

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Developer Spotlight with Denis Gladkikh

From Splunk Engineer to Kubernetes App Builder Denis GladkikhWhat happens when a lifelong developer turns a ...

Governing Enterprise AI, Bringing Cisco Telemetry Home, and More from Splunk Lantern

Splunk Lantern is Splunk’s customer success center that provides practical guidance from Splunk experts on key ...

Agentic SOC Triage: Investigating Splunk ES Notables with MCP Server and a Local LLM

The Problem: Too Many Alerts, Too Little Context Security operations teams running Splunk Enterprise Security ...