Hi,guys,in my scenario,a universal forwarder(UF vertion 4.3.2 for aix) monitores about 700 small files, the cpu usage will be more than 60%. I set a parameter 'ignoreOlderThan=1d' to reduce the cpu usage,but once the modtime of these files changed I want these updated data collected. After this configuration,the data updated in these small files after 23:59:59 till next day will not be collected by splunk agent. So I wanna ask below questions:
1.The mechanism of 'ignoreOlderThan',not up to the file's modtime,but the continuous time that splunk forwarder monitored it since the agent start? Otherwise,why the updated data will be ignored?
2.I did some test,I created a file at 10:00am ,and I set 'ignoreOlderThan=60s' in inputs.conf to monitor it, then I start the agent at 10:05am, this file was still indexed.
I know in the instruction document,it sayes 'ignoreOldThan' checking file by the modtime, but in my case it doesn't work. I haven't tried the current version of splunkforwarder.
The splunk documentation for inputs.conf says, "A file whose modtime falls outside this time window when seen for the first time will not be indexed at all." So, if you had set ignoreOlderThan=1d on an input then the first time the forwarder looked at the directory it is set to watch for that input it would ignore all log files older than one day, and further even if their mod time changes to the current day they will still be ignored.
As for your item #2 using 60s... Sounds like the file you created should have been ignored, unless something updated it within the 60s window!