Splunk Log File Monitoring

[SanjayReddy]

Hi Folks,

We have log file monitoring of one of the text file , and that text file getting updated once in a week. Then Splunk reads the data from that file.

Today we had faced a situation , where log file updated with todays data but not logs were sent to Splunk.

we verified in splunkd.log and didn't find any info related to that specific log file, and Splunk UF connected to HF and everything  is working fine and other data was flowing to Splunk as usal.

However after Splunk restart data sent to splunk,

I was wondering if log file is not getting updated for some time , will Splunk ignores the file from monitoring until restart?.

and we have stanza ignoreOlderthan set to 5d , is this something to do with> .

we are aware that ignoreOlderthan used to lgnore logs data older than specified  time, just wanted to make sure this is not that case.

[somesoni2]

Most probably ignoreOlderthan is the culprit here. Splunk may have got restarted and found the file to be older than 5 days and ignored it (put it in the "ignored" list). It'll stay ignored even after new data is being added. Only restart will make it re-evaluated its file monitoring list and data got ingested.

If the data is updated once every 7 days, keep your ignoreOlderthan match that. 

What kind of updates does the file get, new data gets appended OR it's completed re-written?

[SanjayReddy]

Hi @somesoni2 

Thank you for your explanation. will increase ignoreOlderthan time to match with thelog time update.

regarding your question about log file update , each time log file will be updated with new data , replacing old data in file.