Getting Data In

Logging thousend files via Splunk Forwarder causes high CPU load

tfechner
Path Finder

Hi there,

we have a oracle logging directory with thousend .aud files for logging to Splunk.
Each day over 700 new files will be created.
We experience a heavy workload on the system caused by the splunkd process.

We think splunkd monitores ALL files and after some weeks a hugh bunch of filemonitoring threads are occuping the CPU.

How can we tell splunk not to monitor already indexed files and only have a look on new created. The closed file will never be changed anymore.

Our inputs.conf:

[monitor:///oracle/Q*/trace/audit/*.aud]
sourcetype=oracle:audit:text
whitelist = \w.+.aud
ignoreOlderThan=7d
index=oracle_sap
disabled = false
0 Karma

ddrillic
Ultra Champion

What's the forwarder version? - Universal Forwarder Using High CPU?

0 Karma

tfechner
Path Finder

we use the newst one 7.1.2.X

0 Karma

amiftah
Communicator

I think you have to create your own script to delete/move/rename the indexed files.

0 Karma

sudosplunk
Motivator

Hi,

How are your new files named? Any thing to differentiate new and old.

0 Karma

tfechner
Path Finder

fielname_structure:
AppID_OracleID_timestamp.aud
with:
appid= P56
OracleID: 53457673
time: 2018073134756825434785

0 Karma

sudosplunk
Motivator

The naming doesn't seem to be helpful. Since new files are created every day, decrease ignoreOlderThan to 2 or 3 days. This can reduce load.

0 Karma

tfechner
Path Finder

fielname_structure:
AppID_OracleID_timestamp.aud
with:
appid= P56
OracleID: 53457673
time: 2018073134756825434785

0 Karma
Get Updates on the Splunk Community!

Get ready to show some Splunk Certification swagger at .conf24!

Dive into the deep end of data by earning a Splunk Certification at .conf24. We're enticing you again this ...

Built-in Service Level Objectives Management to Bridge the Gap Between Service & ...

Now On-Demand Join us to learn more about how you can leverage Service Level Objectives (SLOs) and the new ...

Database Performance Sidebar Panel Now on APM Database Query Performance & Service ...

We’ve streamlined the troubleshooting experience for database-related service issues by adding a database ...