- Splunk Answers
- :
- Splunk Administration
- :
- Getting Data In
- :
- Issues of 'ignoreOlderThan' in inputs.conf settin...
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark Topic
- Subscribe to Topic
- Mute Topic
- Printer Friendly Page
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Issues of 'ignoreOlderThan' in inputs.conf setting
Hi,guys,in my scenario,a universal forwarder(UF vertion 4.3.2 for aix) monitores about 700 small files, the cpu usage will be more than 60%. I set a parameter 'ignoreOlderThan=1d' to reduce the cpu usage,but once the modtime of these files changed I want these updated data collected. After this configuration,the data updated in these small files after 23:59:59 till next day will not be collected by splunk agent. So I wanna ask below questions:
1.The mechanism of 'ignoreOlderThan',not up to the file's modtime,but the continuous time that splunk forwarder monitored it since the agent start? Otherwise,why the updated data will be ignored?
2.I did some test,I created a file at 10:00am ,and I set 'ignoreOlderThan=60s' in inputs.conf to monitor it, then I start the agent at 10:05am, this file was still indexed.
I know in the instruction document,it sayes 'ignoreOldThan' checking file by the modtime, but in my case it doesn't work. I haven't tried the current version of splunkforwarder.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
The splunk documentation for inputs.conf says, "A file whose modtime falls outside this time window when seen for the first time will not be indexed at all." So, if you had set ignoreOlderThan=1d on an input then the first time the forwarder looked at the directory it is set to watch for that input it would ignore all log files older than one day, and further even if their mod time changes to the current day they will still be ignored.
As for your item #2 using 60s... Sounds like the file you created should have been ignored, unless something updated it within the 60s window!
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
It would be (and should have been) a nice feature. We have test environments with data is not so frequent and we do lose some data frequently due to this.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
From how I understand the setting to work (im still on 4.3 UF) it checks the modtime during startup of the splunk service. Would be nice for it to periodically.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
It's happening with UF 5.0.5 version as well. Is this a bug and if yes, is it fixed in any release after 5.0.5?
Introducing the 2024 SplunkTrust!
Introducing the 2024 Splunk MVPs!
Splunk Custom Visualizations App End of Life
