topic Re: Excessive logging activity in Getting Data In

Excessive logging activity

mike7860 — Mon, 17 Dec 2012 19:20:19 GMT

I see that the source file splunkd.log is logging excessively. When I look into the diagnostics, I find that my search peers are the originators of these log files. Could anyone suggest how do I minimize the logging activity from splunkd.log source file?

Re: Excessive logging activity

martin_mueller — Mon, 17 Dec 2012 21:26:50 GMT

In case you're worried about license volume - internal logging of Splunk does not count towards your daily allowance.

Re: Excessive logging activity

lguinn2 — Mon, 17 Dec 2012 22:34:49 GMT

Also, the _internal index (where splunkd.log goes) has a maximum size, usually 500 Gb. So the index will not exceed this size; it will roll off the oldest events instead.

Maybe the most important question is - what's in the splunkd.log? It is usually very active, yes, but is it reporting a problem?

Re: Excessive logging activity

bfernandez — Thu, 20 Dec 2012 14:54:47 GMT

In the other hand you can see and change the default system loggings at Manager > System settings > System logging.

What Splunk logs about itself
http://docs.splunk.com/Documentation/Splunk/5.0/Troubleshooting/WhatSplunklogsaboutitself

Re: Excessive logging activity

bfernandez — Thu, 20 Dec 2012 14:54:48 GMT

You can handle the volume of internal indices defining a specified retention policy for them.

Retention policies.
http://wiki.splunk.com/Deploy:BucketRotationAndRetention

Re: Excessive logging activity

gfuente — Thu, 20 Dec 2012 15:05:38 GMT

Also Is it possible to configure Splunk to compress log files so you can reduce the size of the logs files

Re: Excessive logging activity

Drainy — Thu, 20 Dec 2012 15:58:15 GMT

Another alternative, splunkd won't log a massive amount (in comparison). You could reduce the size of the _internal index instead to reduce how much Splunk logs of itself (which can cause a lot of disk usage).

Also, its worth pointing out that the other answer that mentions changing logging levels, the UI will only change it for that instance of Splunk running - when you restart it will reset the logging levels (its just for debugging really).

There are ways to permenantly change the logging levels but you really don't want to, I've always found Splunk to log just the right amount (you may not think so now, but just wait until something breaks 😉 ) and I usually end up increasing the log level if anything.

If you do want to make permenant changes then have a look at;
http://docs.splunk.com/Documentation/Splunk/5.0.1/Troubleshooting/Enabledebuglogging

Re: Excessive logging activity

Dark_Ichigo — Tue, 26 Feb 2013 02:29:01 GMT

I think its all defined in log.cfg, where you can have 5 splunkd.log each with a maximum size of 24mb.