Getting Data In
Highlighted

retention policy on log files

Explorer

Hi, I want to implement retention policy on log files, in the doc https://docs.splunk.com/Documentation/Splunk/8.0.3/Troubleshooting/Enabledebuglogging they didn't mention such a configuration, there is only how to configure the maximum size of a log file (configuration of log-local.cfg), i want this configuration be applied to log files who live in SPLUNK_HOME/var/log, is there any workaround to do so ?

Labels (1)
0 Karma
Highlighted

Re: retention policy on log files

Legend

Hi @marone,
you can set a retention policy for each index (default is 6 years).
you can do this settinf the parameter frozenTimePeriodInSecs in indexes.conf.

You can find more infos at https://docs.splunk.com/Documentation/Splunk/8.0.4/Indexer/Setaretirementandarchivingpolicy

Ciao.
Giuseppe

Highlighted

Re: retention policy on log files

Explorer

thanks for reply, i's true what you are saying but your answer is applied to bucket stored in $SPLUNK_HOME/var/lib/splunk, but the information wil still in log files who lives in SPLUNK_HOME/var/log as I said in my question, i want to apply retention policy on those logs (configure log-local.cfg if it's possible or another way), or applying retention policy as you said for index will automaitcally delete data from logs (aka from SPLUNK_HOME/var/log) ?

0 Karma
Highlighted

Re: retention policy on log files

Legend

Hi @marone,
let me understand: you're speaking of Splunk log files (e.g. splunkd.log.1 2 3 4 5) is it correct?

After they are written on disk, they are read by Splunk and stored in _internal index, this means that you can delete the splunkd.log.1 2 3 4 5 files.

If instead you want to manage retention on internal index, you have to change the frozenTimePeriodInSecs parameter in $SPLUNKHOME/system/local/indexes.conf, if not present create it copying the _internal stanza in default folder.

Ciao.
Giuseppe

0 Karma
Highlighted

Re: retention policy on log files

Explorer

thanks i see what you mean

0 Karma
Highlighted

Re: retention policy on log files

Path Finder

You can add below frozenTimePeriodInSecs in your index stanza.
Example: If you want to retain the logs for 1 day then frozenTimePeriodInSecs = 86400

[_internal]

frozenTimePeriodInSecs = 86400

View solution in original post

Highlighted

Re: retention policy on log files

Explorer

thanks for reply, i's true what you are saying but your answer is applied to bucket stored in $SPLUNK_HOME/var/lib/splunk, but the information wil still in log files who lives in SPLUNK_HOME/var/log as I said in my question, i want to apply retention policy on those logs (configure log-local.cfg if it's possible or another way), or applying retention policy as you said for index will automaitcally delete data from logs (aka from SPLUNK_HOME/var/log) ?

0 Karma
Highlighted

Re: retention policy on log files

Path Finder

All logs under SPLUNK_HOME/var/log will be ingested into _internal index.
Events in this index are kept for 6 years by default.
If you want to change the retention period, you can modify the same for frozenTimePeriodInSecs in indexes.conf for _internal index. Hope this helps and please accept if this helps.

Below is the example:

[internal]
homePath = $SPLUNK
DB/internaldb/db
coldPath = $SPLUNK
DB/internaldb/colddb
thawedPath = $SPLUNK
DB/_internaldb/thaweddb
maxDataSize = 100
frozenTimePeriodInSecs = 2419200

0 Karma
Highlighted

Re: retention policy on log files

Explorer

yes indeed, i was looking for a parameter in log-config.cfg like frozenTimePeriodInSecs but i guess there is not, instead of that configure frozenTimePeriodInSecs in indexes.conf plus reduce amount of file size for all log files will do the job for a good retention policy. Thanks for mentioning that most of logs files are ingested to _internal index

0 Karma
Highlighted

Re: retention policy on log files

Motivator

Hello @marone,

which log files do you have in $SPLUNK_HOME/etc ? Most of logs are under $SPLUNK_HOME/var/log

0 Karma