How to delete some indexed data and free up disk space?

[mahesh_ravji1]

Hi There,

We have an index which is approx. 250GB in size. After change in requirements we no longer require approx. half of the data. I would like to delete approx. half of the indexed data and free up the disk space in the file system. How would I go about achieving this?

Thanks

[ChrisG]

There is a complete topic on this in the documentation: Remove indexes and indexed data. It explains the four main options:

• Delete events from subsequent searches.
• Remove all data from one or more indexes.
• Remove or disable an entire index.
• Delete older data, based on a retirement policy.

[mahesh_ravji1]

Unfortunately none of these options suit our needs. Also cannot use the retirement policy as the data to be deleted spans ‘All Time’.

[chanfoli]

Hopefully some of your older data is in separate warm or cold buckets from data you want to keep, look in your db directory for the index and then read this document:

http://docs.splunk.com/Documentation/Splunk/6.1.4/Indexer/HowSplunkstoresindexes

Then assuming your config is set to roll to warm and cold frequently enough (which you will know by finding or not finding appropriately named buckets), identify old bucket directories and remove them from the filesystem.

[chanfoli]

This of course is assuming that the data you want to keep is newer that the data you no longer need. If they are mixed together in the same index and the value to you is not time based, then you can either reindex the data you want from original source into a new index, or look into the collect command if you don't have the original sources, as you will need to search for the data you want to keep and collect it into a new index, so you can clean or drop the index containing the data you do not want.

[mahesh_ravji1]

Unfortunately the data to be deleted spans 'All Time'. I'll look into the collect command...