Getting Data In

Issues of 'ignoreOlderThan' in inputs.conf setting


Hi,guys,in my scenario,a universal forwarder(UF vertion 4.3.2 for aix) monitores about 700 small files, the cpu usage will be more than 60%. I set a parameter 'ignoreOlderThan=1d' to reduce the cpu usage,but once the modtime of these files changed I want these updated data collected. After this configuration,the data updated in these small files after 23:59:59 till next day will not be collected by splunk agent. So I wanna ask below questions:
1.The mechanism of 'ignoreOlderThan',not up to the file's modtime,but the continuous time that splunk forwarder monitored it since the agent start? Otherwise,why the updated data will be ignored?
2.I did some test,I created a file at 10:00am ,and I set 'ignoreOlderThan=60s' in inputs.conf to monitor it, then I start the agent at 10:05am, this file was still indexed.
I know in the instruction document,it sayes 'ignoreOldThan' checking file by the modtime, but in my case it doesn't work. I haven't tried the current version of splunkforwarder.

Tags (1)


The splunk documentation for inputs.conf says, "A file whose modtime falls outside this time window when seen for the first time will not be indexed at all." So, if you had set ignoreOlderThan=1d on an input then the first time the forwarder looked at the directory it is set to watch for that input it would ignore all log files older than one day, and further even if their mod time changes to the current day they will still be ignored.

As for your item #2 using 60s... Sounds like the file you created should have been ignored, unless something updated it within the 60s window!

0 Karma

Revered Legend

It would be (and should have been) a nice feature. We have test environments with data is not so frequent and we do lose some data frequently due to this.

0 Karma


From how I understand the setting to work (im still on 4.3 UF) it checks the modtime during startup of the splunk service. Would be nice for it to periodically.

0 Karma

Revered Legend

It's happening with UF 5.0.5 version as well. Is this a bug and if yes, is it fixed in any release after 5.0.5?

0 Karma
Get Updates on the Splunk Community!

Build Scalable Security While Moving to Cloud - Guide From Clayton Homes

 Clayton Homes faced the increased challenge of strengthening their security posture as they went through ...

Mission Control | Explore the latest release of Splunk Mission Control (2.3)

We’re happy to announce the release of Mission Control 2.3 which includes several new and exciting features ...

Cloud Platform | Migrating your Splunk Cloud deployment to Python 3.7

Python 2.7, the last release of Python 2, reached End of Life back on January 1, 2020. As part of our larger ...