Getting Data In

Logging thousend files via Splunk Forwarder causes high CPU load

Path Finder

Hi there,

we have a oracle logging directory with thousend .aud files for logging to Splunk.
Each day over 700 new files will be created.
We experience a heavy workload on the system caused by the splunkd process.

We think splunkd monitores ALL files and after some weeks a hugh bunch of filemonitoring threads are occuping the CPU.

How can we tell splunk not to monitor already indexed files and only have a look on new created. The closed file will never be changed anymore.

Our inputs.conf:

[monitor:///oracle/Q*/trace/audit/*.aud]
sourcetype=oracle:audit:text
whitelist = \w.+.aud
ignoreOlderThan=7d
index=oracle_sap
disabled = false
0 Karma

Ultra Champion

What's the forwarder version? - Universal Forwarder Using High CPU?

0 Karma

Path Finder

we use the newst one 7.1.2.X

0 Karma

Communicator

I think you have to create your own script to delete/move/rename the indexed files.

0 Karma

Motivator

Hi,

How are your new files named? Any thing to differentiate new and old.

0 Karma

Path Finder

fielnamestructure:
AppID
OracleID_timestamp.aud
with:
appid= P56
OracleID: 53457673
time: 2018073134756825434785

0 Karma

Motivator

The naming doesn't seem to be helpful. Since new files are created every day, decrease ignoreOlderThan to 2 or 3 days. This can reduce load.

0 Karma

Path Finder

fielnamestructure:
AppID
OracleID_timestamp.aud
with:
appid= P56
OracleID: 53457673
time: 2018073134756825434785

0 Karma