Archiving of Indexes Not Working

[jkfierro]

We are running Splunk 6.0.1 on a Centos Linux virtual machine.

The splunk application and indexes reside on a 200 GB disk. Of this 200 GB, hot/warm indexes consume about 165 GB.

Periodically, we get the message in splunkd.log (of course bucket name changes each time):

INFO BucketMover - will attempt to freeze bkt='/opt/splunk/var/lib/splunk/defaultdb/db/db_1363632727_1363632204_23' because maxTotalDataSize=178257920000 bytes, currentSize=178260974557 bytes

So, once the total index size reaches 166 GB, it deletes the oldest indexes. But we don't want this. We want to archive this data. Note: I have no idea where this setting "maxTotalDataSize" is coming from at the moment.

I have set coldToFrozenDir in the hopes that it will archive the indexes before deleting them. But it's not archving them at all. (Yes I restarted Splunk to take effect).

What do you think my next step should be? Thanks.

[MuS]

Hi jkfierro,

this topic needs a lot of reading docs and wiki to be fully understood. But there are some very good examples like this:

• http://docs.splunk.com/Documentation/Splunk/6.0/Indexer/Automatearchiving
• http://docs.splunk.com/Documentation/Splunk/6.0/Indexer/Setaretirementandarchivingpolicy
• http://wiki.splunk.com/Deploy:BucketRotationAndRetention
• http://wiki.splunk.com/Deploy:UnderstandingBuckets

hope after that, you get the archiving do what it should for you.

cheers, MuS

[jkfierro]

I have already reviewed documentation on this. It was not clear to me what my particular issue is and how Splunk wants to behave in handling the indexes/archiving.

[aelliott]

you can actually change your maxTotalDataSize in the UI and change your Frozen archive path.
Settings -> Indexes, then click on an index.
I would first verify that the path is there.