Getting Data In

How to delete some indexed data and free up disk space?

mahesh_ravji1
Explorer

Hi There,

We have an index which is approx. 250GB in size. After change in requirements we no longer require approx. half of the data. I would like to delete approx. half of the indexed data and free up the disk space in the file system. How would I go about achieving this?

Thanks

Tags (3)
0 Karma

ChrisG
Splunk Employee
Splunk Employee

There is a complete topic on this in the documentation: Remove indexes and indexed data. It explains the four main options:

  • Delete events from subsequent searches.
  • Remove all data from one or more indexes.
  • Remove or disable an entire index.
  • Delete older data, based on a retirement policy.
0 Karma

mahesh_ravji1
Explorer

Unfortunately none of these options suit our needs. Also cannot use the retirement policy as the data to be deleted spans ‘All Time’.

0 Karma

chanfoli
Builder

Hopefully some of your older data is in separate warm or cold buckets from data you want to keep, look in your db directory for the index and then read this document:

http://docs.splunk.com/Documentation/Splunk/6.1.4/Indexer/HowSplunkstoresindexes

Then assuming your config is set to roll to warm and cold frequently enough (which you will know by finding or not finding appropriately named buckets), identify old bucket directories and remove them from the filesystem.

0 Karma

chanfoli
Builder

This of course is assuming that the data you want to keep is newer that the data you no longer need. If they are mixed together in the same index and the value to you is not time based, then you can either reindex the data you want from original source into a new index, or look into the collect command if you don't have the original sources, as you will need to search for the data you want to keep and collect it into a new index, so you can clean or drop the index containing the data you do not want.

0 Karma

mahesh_ravji1
Explorer

Unfortunately the data to be deleted spans 'All Time'. I'll look into the collect command...

0 Karma
Get Updates on the Splunk Community!

Splunk App for Anomaly Detection End of Life Announcment

Q: What is happening to the Splunk App for Anomaly Detection?A: Splunk is officially announcing the ...

Aligning Observability Costs with Business Value: Practical Strategies

 Join us for an engaging Tech Talk on Aligning Observability Costs with Business Value: Practical ...

Mastering Data Pipelines: Unlocking Value with Splunk

 In today's AI-driven world, organizations must balance the challenges of managing the explosion of data with ...