Getting Data In

Why is Splunk not capturing new content in a CSV file input and indexing it?

Explorer

Hi,

Splunk 6.5.0

I have the scenario that I have to import every hour a csv (File A) file from a system which has no network access/connection to the Splunk environment. I've created a csv (File B) file on the Splunk indexer which is defined as data input.

Every hour there is running a process which cuts the contend from File A and writes it to File B. Mostly Splunk is indexing the content but sometimes it happens, that File B is filled with the new content but Splunk doesn't index it.

Has somebody an idea why that happens?

0 Karma
1 Solution

SplunkTrust
SplunkTrust

Does the file have a large header that is the same in every instance? Splunk will look at the first 256 bytes to determine if it's already indexed the file and refuse to ingest it again. Consider changing the crcSalt or initCrcLength settings in inputs.conf.

---
If this reply helps you, an upvote would be appreciated.

View solution in original post

SplunkTrust
SplunkTrust

Does the file have a large header that is the same in every instance? Splunk will look at the first 256 bytes to determine if it's already indexed the file and refuse to ingest it again. Consider changing the crcSalt or initCrcLength settings in inputs.conf.

---
If this reply helps you, an upvote would be appreciated.

View solution in original post

Explorer

Hi, it could happen that there is no change in the header or first 256 bytes.
Regarding your hint, i'll change the size of the initCrcLength to a higher value and check if my problem has gone.

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!