I installed the Splunk universal forwarder (agents) on several clients, running several days.
# pwd
/opt/splunkforwarder/etc
# grep metric log.cfg
# metrics spews a lot of logs, let's not pollute the other files.
appender.metrics=RollingFileAppender
appender.metrics.fileName=${SPLUNK_HOME}/var/log/splunk/metrics.log
appender.metrics.maxFileSize=25000000 # default: 25MB (specified in bytes).
appender.metrics.maxBackupIndex=5
appender.metrics.layout=PatternLayout
appender.metrics.layout.ConversionPattern=%d{%m-%d-%Y %H:%M:%S.%l %z} %-5p %c - %m%n
category.Metrics=INFO,metrics
category.StatusMgr=INFO,metrics
# ls -lctr |grep metric
-rw-------. 1 root root 115789498 Sep 15 17:51 metrics.log.5
-rw-------. 1 root root 110047302 Sep 15 17:51 metrics.log.4
-rw-------. 1 root root 110284563 Sep 15 17:51 metrics.log.3
-rw-------. 1 root root 25926442 Sep 15 17:51 metrics.log.2
-rw-------. 1 root root 82850928 Sep 15 17:51 metrics.log.1
-rw-------. 1 root root 62256009 Sep 16 11:35 metrics.log
Have the setting (max 25MB, and 5 backups), but the rotate log sizes are from 25MB ~ 110MB. Anything wrong and how can I fix it?
I need the rotate log keep the size in 25MB each.
Looking at the set-up here and it looks good -
-rw-------. 1 splnkfwd splnkfwd 24M Feb 4 07:57 metrics.log.5
-rw-------. 1 splnkfwd splnkfwd 24M Feb 5 18:41 metrics.log.4
-rw-------. 1 splnkfwd splnkfwd 24M Feb 7 05:35 metrics.log.3
-rw-------. 1 splnkfwd splnkfwd 24M Feb 8 16:24 metrics.log.2
-rw-------. 1 splnkfwd splnkfwd 24M Feb 10 03:13 metrics.log.1
-rw-------. 1 splnkfwd splnkfwd 21M Feb 11 08:55 metrics.log
-rw-------. 1 splnkfwd splnkfwd 9.3M Feb 11 08:55 splunkd.log
$ grep metric log.cfg
appender.metrics=RollingFileAppender
appender.metrics.fileName=${SPLUNK_HOME}/var/log/splunk/metrics.log
appender.metrics.maxFileSize=25000000 # default: 25MB (specified in bytes).
appender.metrics.maxBackupIndex=5
appender.metrics.layout=PatternLayout
appender.metrics.layout.ConversionPattern=%d{%m-%d-%Y %H:%M:%S.%l %z} %-5p %c - %m%n
category.Metrics=INFO,metrics
category.StatusMgr=INFO,metrics
We also having the same issue.(Though the default is 25MB files are more than 25MB) Were you able to find the root cause?
1.They arent running in debug mode are they? 2. Have you upgraded or re-installed the UFs? (log.cfg will be overwritten. Use log-local.cfg instead.)
Thanks @rroberts. Debug is not enable and no log-local.cfg.
[splunkforwarder]# pwd
/opt/splunkforwarder
[splunkforwarder]# grep -i debug etc/log.cfg
# This file contains the debugging output controls
# Customers can change debugging levels as needed with output going to
[splunkforwarder]# find . -type f |grep log|grep local
[splunkforwarder]#