Splunk Enterprise

How to efficiently delete data from index that has high volume?

architkhanna
Path Finder

Hi Splunkers , 

I have a splunk index with 3 source types corresponding to each ticket types. it has millions of record in last 10 months and we have now started re pulling all the data again due to 2 new fields which client wants to onboard.

Since we do not want to keep the older records which does not have the new fields , We need to find out a way on how to identify the data eligible for deletion. Please note, all tickets have updates more than 1 times up to 50 times as well.


Labels (2)
Tags (2)
0 Karma

PickleRick
SplunkTrust
SplunkTrust

As a general rule - you don't "delete" data from indexes. Once the data is ingested and indexed it stays in the buckets until whole bucket is rolled out to frozen and deleted.

True, there is a "delete" command but it doesn't really delete data from the buckets, it just marks it unaccessible. And in production I wouldn't really use that.

Probably the easiest approach for you would be to drop your index completely, recreate it from scratch and reindex the data (yes, indexing it again will count against your license).

0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud's AI Assistant in Action Series: Auditing Compliance and ...

This is the third post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how to ...

Splunk Community Badges!

  Hey everyone! Ready to earn some serious bragging rights in the community? Along with our existing badges ...

What You Read The Most: Splunk Lantern’s Most Popular Articles!

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...