Is there a need for keeping the _internal index logs past a certain time period?

[michaeler]

Is there a need for keeping the _internal index logs past a certain time period? My _internaldb is pretty large at 218GB total, db - 31, cold - 112, frozen - 75. You can see my current settings below. We have about 140 forwarders reporting to this indexer.

Should I just remove the path to frozen and let them get deleted? Does anyone ever thaw internal logs? If so, what for?

[_internal]
homePath = $SPLUNK_DB\_internaldb\db
coldPath = $SPLUNK_DB\_internaldb\colddb
thawedPath = $SPLUNK_DB\_internaldb\thaweddb
coldToFrozenDir = $SPLUNK_DB\_internaldb\frozendb
frozenTimePeriodInSecs = 5184000
tstatsHomePath = volume:_splunk_summaries\_internaldb\datamodel_summary
maxConcurrentOptimizes = 6
maxWarmDBCount = 60
maxHotSpanSecs = 86400
maxHotBuckets = 8
maxDataSize = auto

[somesoni2]

The _internal logs stores data about the Splunk software itself and often used to troubleshoot an issue with Splunk (or see the trend). In most common cases, keeping the logs for 4 week (or a month) is more than sufficient and thawing is generally not used with internal indexes. So you should be able to adjust your indexes.conf parameters to keep your _internal data for a month/30 days. If you need to retain some specific events for more time OR need to do some trending, setup a summary index and store summarized data for longer period.

[isoutamo]

Hi

it's just like @somesoni2 said. It depends based on your needs. Personally I like to keep those as long as possible online to check what has happened when there are some errors or weird behaviour. Of course you must think about risk if you haven't those in your hands when/If needed. Probably this is not so big issue. 

One option is to archive those thawed files e.g. to tapes and if needed then retrieve those if needed.

Probably biggest use for those are getting better understanding what has happened on splunk environment based on statistics. Of course you could get most of this with summary indexes etc. Which leads that no need to keep those actual files on your hands.

One conf presentation which could help you to figure out how you should proceed https://conf.splunk.com/files/2021/slides/PLA1410C.pdf

From Noob to Ninja: Growing and Managing Splunk Enterprise as a Team of One 

PLA1410C Dan Burras

r. Ismo