Getting Data In

Why is Splunk not capturing new content in a CSV file input and indexing it?

bosch_softtec
Path Finder

Hi,

Splunk 6.5.0

I have the scenario that I have to import every hour a csv (File A) file from a system which has no network access/connection to the Splunk environment. I've created a csv (File B) file on the Splunk indexer which is defined as data input.

Every hour there is running a process which cuts the contend from File A and writes it to File B. Mostly Splunk is indexing the content but sometimes it happens, that File B is filled with the new content but Splunk doesn't index it.

Has somebody an idea why that happens?

0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

Does the file have a large header that is the same in every instance? Splunk will look at the first 256 bytes to determine if it's already indexed the file and refuse to ingest it again. Consider changing the crcSalt or initCrcLength settings in inputs.conf.

---
If this reply helps you, Karma would be appreciated.

View solution in original post

richgalloway
SplunkTrust
SplunkTrust

Does the file have a large header that is the same in every instance? Splunk will look at the first 256 bytes to determine if it's already indexed the file and refuse to ingest it again. Consider changing the crcSalt or initCrcLength settings in inputs.conf.

---
If this reply helps you, Karma would be appreciated.

bosch_softtec
Path Finder

Hi, it could happen that there is no change in the header or first 256 bytes.
Regarding your hint, i'll change the size of the initCrcLength to a higher value and check if my problem has gone.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...