topic Re: Logging thousend files via Splunk Forwarder causes high CPU load in Getting Data In

Logging thousend files via Splunk Forwarder causes high CPU load

tfechner — Fri, 10 Aug 2018 13:59:55 GMT

Hi there,

we have a oracle logging directory with thousend .aud files for logging to Splunk.
Each day over 700 new files will be created.
We experience a heavy workload on the system caused by the splunkd process.

We think splunkd monitores ALL files and after some weeks a hugh bunch of filemonitoring threads are occuping the CPU.

How can we tell splunk not to monitor already indexed files and only have a look on new created. The closed file will never be changed anymore.

Our inputs.conf:

[monitor:///oracle/Q*/trace/audit/*.aud]
sourcetype=oracle:audit:text
whitelist = \w.+.aud
ignoreOlderThan=7d
index=oracle_sap
disabled = false

Re: Logging thousend files via Splunk Forwarder causes high CPU load

sudosplunk — Fri, 10 Aug 2018 14:28:33 GMT

Hi,

How are your new files named? Any thing to differentiate new and old.

Re: Logging thousend files via Splunk Forwarder causes high CPU load

amiftah — Fri, 10 Aug 2018 14:43:48 GMT

I think you have to create your own script to delete/move/rename the indexed files.

Re: Logging thousend files via Splunk Forwarder causes high CPU load

tfechner — Tue, 29 Sep 2020 20:53:40 GMT

fielname_structure:
AppID_OracleID_timestamp.aud
with:
appid= P56
OracleID: 53457673
time: 2018073134756825434785

Re: Logging thousend files via Splunk Forwarder causes high CPU load

tfechner — Tue, 29 Sep 2020 20:53:43 GMT

fielname_structure:
AppID_OracleID_timestamp.aud
with:
appid= P56
OracleID: 53457673
time: 2018073134756825434785

Re: Logging thousend files via Splunk Forwarder causes high CPU load

sudosplunk — Fri, 10 Aug 2018 15:18:54 GMT

The naming doesn't seem to be helpful. Since new files are created every day, decrease ignoreOlderThan to 2 or 3 days. This can reduce load.

Re: Logging thousend files via Splunk Forwarder causes high CPU load

ddrillic — Sat, 11 Aug 2018 22:55:01 GMT

What's the forwarder version? - Universal Forwarder Using High CPU?

Re: Logging thousend files via Splunk Forwarder causes high CPU load

tfechner — Mon, 13 Aug 2018 07:39:52 GMT

we use the newst one 7.1.2.X