Join the Conversation
- Find Answers
- :
- Splunk Administration
- :
- Getting Data In
- :
- Logging thousend files via Splunk Forwarder causes...
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark Topic
- Subscribe to Topic
- Mute Topic
- Printer Friendly Page
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Logging thousend files via Splunk Forwarder causes high CPU load
Hi there,
we have a oracle logging directory with thousend .aud files for logging to Splunk.
Each day over 700 new files will be created.
We experience a heavy workload on the system caused by the splunkd process.
We think splunkd monitores ALL files and after some weeks a hugh bunch of filemonitoring threads are occuping the CPU.
How can we tell splunk not to monitor already indexed files and only have a look on new created. The closed file will never be changed anymore.
Our inputs.conf:
[monitor:///oracle/Q*/trace/audit/*.aud]
sourcetype=oracle:audit:text
whitelist = \w.+.aud
ignoreOlderThan=7d
index=oracle_sap
disabled = false
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
What's the forwarder version? - Universal Forwarder Using High CPU?
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
we use the newst one 7.1.2.X
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I think you have to create your own script to delete/move/rename the indexed files.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
How are your new files named? Any thing to differentiate new and old.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
fielname_structure:
AppID_OracleID_timestamp.aud
with:
appid= P56
OracleID: 53457673
time: 2018073134756825434785
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
The naming doesn't seem to be helpful. Since new files are created every day, decrease ignoreOlderThan to 2 or 3 days. This can reduce load.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
fielname_structure:
AppID_OracleID_timestamp.aud
with:
appid= P56
OracleID: 53457673
time: 2018073134756825434785
Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!
Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.
[Puzzles] Solve, Learn, Repeat: Character substitutions with Regular Expressions
Splunk Community Badges!
[Puzzles] Solve, Learn, Repeat: Matching cron expressions
