topic Why is Splunk not ingesting .txt logfiles starting with a numeric value? in Splunk Enterprise

Why is Splunk not ingesting .txt logfiles starting with a numeric value?

dbrooks_CIR — Mon, 28 Mar 2022 21:02:07 GMT

I am trying to bring in some .txt logfiles using Splunk forwarder. There are several logs in the directory, such as Log.txt, 10Log.txt, 20Log.txt, etc. These files are changed daily, and the 10, 20, etclog.txt files are written to daily. So far, I can only get Splunk to ingest the Log.txt file and nothing else. My inputs.conf file is currently as below. I have tried to monitor just *.txt with the same results. Only Log.txt is read/ingested.

[monitor://E:\Logs\CIR_Remote\*Log.txt]
disabled = false
sourcetype = LOG4NET
index = log4net
initCrcLength=1024

any input would be appreciated!

Re: Splunk not ingesting .txt logfiles starting with a numeric value.

somesoni2 — Mon, 28 Mar 2022 20:22:27 GMT

Check if first few lines/characters of all those files (Log.txt and <N>Log.txt) have some common header/characters. If they've some common header/characters, Splunk will think that they're rolled log files. To resolve it, you'll need to adjust initCrcLength value to be slighly more that the length of those common header/characters.

Re: Splunk not ingesting .txt logfiles starting with a numeric value.

dbrooks_CIR — Mon, 28 Mar 2022 20:50:06 GMT

No, the files are not the same. They start with a datetime stamp, and then include error information. They have never been ingested at all, always skipped for some reason.

Re: Why is Splunk not ingesting .txt logfiles starting with a numeric value?

PickleRick — Mon, 28 Mar 2022 21:05:14 GMT

I don't know why (didn't bother to investigate it) but I noticed that sometimes specifying wildcard path to files doesn't work whereas specifying whole directory with a whitelist matching only set of files does.

Unfortunately, you can't specify two different sets of files from the same directory since each input stanza must be unique.

Anyway, check your config with

splunk btool inputs list --debug

and verify list of monitored files with

splunk list monitor