Getting Data In

Why is Splunk not capturing new content in a CSV file input and indexing it?

bosch_softtec
Path Finder

Hi,

Splunk 6.5.0

I have the scenario that I have to import every hour a csv (File A) file from a system which has no network access/connection to the Splunk environment. I've created a csv (File B) file on the Splunk indexer which is defined as data input.

Every hour there is running a process which cuts the contend from File A and writes it to File B. Mostly Splunk is indexing the content but sometimes it happens, that File B is filled with the new content but Splunk doesn't index it.

Has somebody an idea why that happens?

0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

Does the file have a large header that is the same in every instance? Splunk will look at the first 256 bytes to determine if it's already indexed the file and refuse to ingest it again. Consider changing the crcSalt or initCrcLength settings in inputs.conf.

---
If this reply helps you, Karma would be appreciated.

View solution in original post

richgalloway
SplunkTrust
SplunkTrust

Does the file have a large header that is the same in every instance? Splunk will look at the first 256 bytes to determine if it's already indexed the file and refuse to ingest it again. Consider changing the crcSalt or initCrcLength settings in inputs.conf.

---
If this reply helps you, Karma would be appreciated.

bosch_softtec
Path Finder

Hi, it could happen that there is no change in the header or first 256 bytes.
Regarding your hint, i'll change the size of the initCrcLength to a higher value and check if my problem has gone.

0 Karma
Get Updates on the Splunk Community!

Automatic Discovery Part 1: What is Automatic Discovery in Splunk Observability Cloud ...

If you’ve ever deployed a new database cluster, spun up a caching layer, or added a load balancer, you know it ...

Real-Time Fraud Detection: How Splunk Dashboards Protect Financial Institutions

Financial fraud isn't slowing down. If anything, it's getting more sophisticated. Account takeovers, credit ...

Splunk + ThousandEyes: Correlate frontend, app, and network data to troubleshoot ...

 Are you tired of troubleshooting delays caused by siloed frontend, application, and network data? We've got a ...