https://community.splunk.com/t5/Getting-Data-In/Is-there-a-way-set-CPU-and-Memory-consumption-for-splunkd/m-p/315911#M59114 <P>Related and also from today - <A href="https://answers.splunk.com/answers/567219/how-can-we-minimize-the-memory-footprint-of-a-forw.html">How can we minimize the memory footprint of a forwarder?</A></P> Fri, 01 Sep 2017 16:47:45 GMT ddrillic 2017-09-01T16:47:45Z https://community.splunk.com/t5/Getting-Data-In/Is-there-a-way-set-CPU-and-Memory-consumption-for-splunkd/m-p/315910#M59113 <P>We have more than 3000+ forwarders in our environment. Few weeks back unix team has published a report showing all the top process that consume more cpu and memory usage. </P> <P>Splunkd was among the top 3. We need to somehow restrict splunkd from taking up so much usage. on few them the report was showing splunkd had taken more than 90% mem usage.</P> <P>Please suggest a way to mitigate this issue. </P> Fri, 01 Sep 2017 15:52:34 GMT https://community.splunk.com/t5/Getting-Data-In/Is-there-a-way-set-CPU-and-Memory-consumption-for-splunkd/m-p/315910#M59113 sandyIscream 2017-09-01T15:52:34Z https://community.splunk.com/t5/Getting-Data-In/Is-there-a-way-set-CPU-and-Memory-consumption-for-splunkd/m-p/315911#M59114 <P>Related and also from today - <A href="https://answers.splunk.com/answers/567219/how-can-we-minimize-the-memory-footprint-of-a-forw.html">How can we minimize the memory footprint of a forwarder?</A></P> Fri, 01 Sep 2017 16:47:45 GMT https://community.splunk.com/t5/Getting-Data-In/Is-there-a-way-set-CPU-and-Memory-consumption-for-splunkd/m-p/315911#M59114 ddrillic 2017-09-01T16:47:45Z https://community.splunk.com/t5/Getting-Data-In/Is-there-a-way-set-CPU-and-Memory-consumption-for-splunkd/m-p/315912#M59115 <P>The CPU and memory usage for splunkd on a forwarder is directly related to the amount of work it has to do.</P> <P>One of the most common reasons for a busy splunkd is that the forwarder is monitoring a lot of files. Even if the files are inactive, it still requires resources to monitor them. If you run <CODE>splunk list monitor</CODE> on a forwarder, you may be surprised at the list of files that are being watched.</P> <P>If that is the case, you should set up regular log file rotation to remove older files from the production servers where the forwarders run. Another alternative is to use the <CODE>ignoreOlderThan</CODE> setting in inputs.conf. See <A href="https://docs.splunk.com/Documentation/SplunkCloud/6.6.1/Data/Monitorfilesanddirectorieswithinputs.conf">Monitor files and directories</A> for more details, but be careful not to exclude files that might be updated.</P> <P>You should find that reducing the number of files monitored will reduce the memory footprint and the cpu usage.</P> <P>PS. Greater than 90% CPU usage for splunkd definitely indicates a problem. If not this, then something else is misconfigured. The only time that splunkd <EM>might</EM> need this much CPU is if it is monitoring a very high volume input... and that may require special attention.</P> Fri, 01 Sep 2017 17:42:44 GMT https://community.splunk.com/t5/Getting-Data-In/Is-there-a-way-set-CPU-and-Memory-consumption-for-splunkd/m-p/315912#M59115 lguinn2 2017-09-01T17:42:44Z https://community.splunk.com/t5/Getting-Data-In/Is-there-a-way-set-CPU-and-Memory-consumption-for-splunkd/m-p/315913#M59116 <P>Yes this was the exact thing we are looking for. On that particular system splunk indeed is monitoring a lot of files. Around 100k different source paths, though there filesystem do have a log rotation policy but maybe the no of files is the problem here. </P> Sat, 02 Sep 2017 11:56:58 GMT https://community.splunk.com/t5/Getting-Data-In/Is-there-a-way-set-CPU-and-Memory-consumption-for-splunkd/m-p/315913#M59116 sandyIscream 2017-09-02T11:56:58Z