Hi, I want to implement retention policy on log files, in the doc https://docs.splunk.com/Documentation/Splunk/8.0.3/Troubleshooting/Enabledebuglogging they didn't mention such a configuration, there is only how to configure the maximum size of a log file (configuration of log-local.cfg), i want this configuration be applied to log files who live in SPLUNK_HOME/var/log
, is there any workaround to do so ?
You can add below frozenTimePeriodInSecs in your index stanza.
Example: If you want to retain the logs for 1 day then frozenTimePeriodInSecs = 86400
[_internal]
frozenTimePeriodInSecs = 86400
Hello @marone,
which log files do you have in $SPLUNK_HOME/etc
? Most of logs are under $SPLUNK_HOME/var/log
thanks for your comment, yes a typo error logs are in $SPLUNK_HOME/var/log
and the log-local.cfg are in $SPLUNK_HOME/etc
i modified it
@marone, I think you asking about log file rotation and autodeletion, right? This is configured under log.cfg:
appender.rootAppender=ConsoleAppender
appender.rootAppender.layout=PatternLayout
appender.rootAppender.layout.ConversionPattern=%d{%m-%d-%Y %H:%M:%S.%l %z} %-5p %c - %m%n
# if these log files are getting too big for your liking, turn down the maxFileSize.
# it's best to not make them too small, however, because these logs can be very
# useful in troubleshooting.
appender.A1=RollingFileAppender
appender.A1.fileName=${SPLUNK_HOME}/var/log/splunk/splunkd.log
appender.A1.maxFileSize=25000000 # default: 25MB (specified in bytes).
appender.A1.maxBackupIndex=5
appender.A1.layout=PatternLayout
as you can see, for most log files that is max 5 log files x 25MB. More for license log.
Is this the info that you looked for?
yes i saw the doc was looking for a parameter like frozenTimePeriodInSecs
in indexes.conf, but i think decreasing maxFilesize and number of backup index plus configuring frozenTimePeriodInSecs
for _internal
will result to a good retention policy i guess
You can add below frozenTimePeriodInSecs in your index stanza.
Example: If you want to retain the logs for 1 day then frozenTimePeriodInSecs = 86400
[_internal]
frozenTimePeriodInSecs = 86400
thanks for reply, i's true what you are saying but your answer is applied to bucket stored in $SPLUNK_HOME/var/lib/splunk
, but the information wil still in log files who lives in SPLUNK_HOME/var/log
as I said in my question, i want to apply retention policy on those logs (configure log-local.cfg if it's possible or another way), or applying retention policy as you said for index will automaitcally delete data from logs (aka from SPLUNK_HOME/var/log
) ?
All logs under SPLUNK_HOME/var/log will be ingested into _internal index.
Events in this index are kept for 6 years by default.
If you want to change the retention period, you can modify the same for frozenTimePeriodInSecs in indexes.conf for _internal index. Hope this helps and please accept if this helps.
Below is the example:
[_internal]
homePath = $SPLUNK_DB/_internaldb/db
coldPath = $SPLUNK_DB/_internaldb/colddb
thawedPath = $SPLUNK_DB/_internaldb/thaweddb
maxDataSize = 100
frozenTimePeriodInSecs = 2419200
yes indeed, i was looking for a parameter in log-config.cfg like frozenTimePeriodInSecs but i guess there is not, instead of that configure frozenTimePeriodInSecs in indexes.conf plus reduce amount of file size for all log files will do the job for a good retention policy. Thanks for mentioning that most of logs files are ingested to _internal index
Hi @marone,
you can set a retention policy for each index (default is 6 years).
you can do this settinf the parameter frozenTimePeriodInSecs
in indexes.conf.
You can find more infos at https://docs.splunk.com/Documentation/Splunk/8.0.4/Indexer/Setaretirementandarchivingpolicy
Ciao.
Giuseppe
thanks for reply, i's true what you are saying but your answer is applied to bucket stored in $SPLUNK_HOME/var/lib/splunk
, but the information wil still in log files who lives in SPLUNK_HOME/var/log
as I said in my question, i want to apply retention policy on those logs (configure log-local.cfg if it's possible or another way), or applying retention policy as you said for index will automaitcally delete data from logs (aka from SPLUNK_HOME/var/log
) ?
Hi @marone,
let me understand: you're speaking of Splunk log files (e.g. splunkd.log.1 2 3 4 5) is it correct?
After they are written on disk, they are read by Splunk and stored in _internal index, this means that you can delete the splunkd.log.1 2 3 4 5 files.
If instead you want to manage retention on _internal index, you have to change the frozenTimePeriodInSecs
parameter in $SPLUNK_HOME/system/local/indexes.conf, if not present create it copying the _internal stanza in default folder.
Ciao.
Giuseppe
thanks i see what you mean