Getting Data In

Splunk suddenly stops ingesting data.


Hi All,

I have a scripted output file that splunk is ingesting via a heavy forwarder.

Since last few weeks, I am facing an issue like suddenly splunk stops ingesting the data eventhough the script is writing the data to the output file.

The script is configured to run every 2 minutes, and remove the previous data and write the new data onto it.

When I check the internal logs, I get the below error :

03-18-2020 12:50:01.521 +0000 ERROR TailReader - Ignoring path="/tmp/splunkDataFiles/labSanityCheck.txt" due to: Bug: tried to check/configure STData processing but have no pending metadata.

03-18-2020 12:50:01.517 +0000 ERROR TailReader - failed to compute crc for /tmp/splunkDataFiles/labSanityCheck.txt (method: 0, hint: No such file or directory).

As per some previous answers to this similar problem, I updated the CHARSET=AUTO, but that did not help.

Can somebody suggest anything regarding this issue?????

0 Karma


I am also having this same issue. In my case the file being read contains data coming in via syslog and written to disk. In the past I have deleted the files and restarted my Universal forwarder, which worked for a while, but eventually the logs would stop and I would see the crc errors in my splunk logs.

0 Karma

Esteemed Legend

We see this problem all the time and it is usually due to there being way too many files co-resident with the files that you are monitoring. This typically happens because there is no housekeeping, or very languishing policy for deleting the files as they rotate. Yes, even if you are not monitoring the rotated files because they do not match the pattern in your [monitor...] stanza, as they pile up, they will eventually slow the forwarder down to a crawl. It usually starts when you have hundreds of files and you are crippled by the time you get to thousands. If you cannot delete the files that are way old and done, then you can create soft links to fresh files in another directory. Let me know if you need details on how to do that.

0 Karma


Apologies for the delayed response...

The script I am talking about here deletes the existing data in the output file and overwrites the file with new data.

0 Karma


the script output can be buffered and not flushing to the file system immedially. This especially the case if your script produces a small output only. Try to google "flush stdout your_script_language"

0 Karma
Get Updates on the Splunk Community!

Registration for Splunk University is Now Open!

Are you ready for an adventure in learning?   Brace yourselves because Splunk University is back, and it's ...

Splunkbase | Splunk Dashboard Examples App for SimpleXML End of Life

The Splunk Dashboard Examples App for SimpleXML will reach end of support on Dec 19, 2024, after which no new ...

Understanding Generative AI Techniques and Their Application in Cybersecurity

Watch On-Demand Artificial intelligence is the talk of the town nowadays, with industries of all kinds ...