Getting Data In

Are there any plans to add compaction of the internal index databases? and one extra related question.

mce128
Explorer

Hi,

I was just curious to know if adding the ability to compact the index databases is on the product timeline. It would be very nice to be able to compact the indexes of deleted data when neeeded. Albiet, that in a normally running system, it would only be used on a by exception basis as generally one is not actually deleting event records.

However, there are times when this would be a real life saver instead of having to fully remove and index and then re-index all of your files (if you even have them all for the time period.) After all, re-indexing everything with enough data could take days and throw you way over your license limits and thereby get your ability to search, etc locked out.

One other related question: Can you drop in an index from another host with everything intact, so that a particular host can take over that index as well? Or perhaps, process the index from another host into a newly created index?

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

Have you looked at what the Splunk coldToFrozenDir setting does? it will in fact archive Splunk data to a specific path, and those indexes can be rebuilt/thawed at no cost to license, though it will require time and CPU to rebuild. There is really no other compaction necessary or possible, unless you have been using the "delete" command a lot.

Get Updates on the Splunk Community!

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...