Splunk Enterprise

How to efficiently delete data from index that has high volume?

architkhanna
Path Finder

Hi Splunkers , 

I have a splunk index with 3 source types corresponding to each ticket types. it has millions of record in last 10 months and we have now started re pulling all the data again due to 2 new fields which client wants to onboard.

Since we do not want to keep the older records which does not have the new fields , We need to find out a way on how to identify the data eligible for deletion. Please note, all tickets have updates more than 1 times up to 50 times as well.


Labels (2)
Tags (2)
0 Karma

PickleRick
SplunkTrust
SplunkTrust

As a general rule - you don't "delete" data from indexes. Once the data is ingested and indexed it stays in the buckets until whole bucket is rolled out to frozen and deleted.

True, there is a "delete" command but it doesn't really delete data from the buckets, it just marks it unaccessible. And in production I wouldn't really use that.

Probably the easiest approach for you would be to drop your index completely, recreate it from scratch and reindex the data (yes, indexing it again will count against your license).

0 Karma
Get Updates on the Splunk Community!

What's New in Splunk Cloud Platform 9.2.2403?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.2.2403! Analysts can ...

Stay Connected: Your Guide to July and August Tech Talks, Office Hours, and Webinars!

Dive into our sizzling summer lineup for July and August Community Office Hours and Tech Talks. Scroll down to ...

Edge Processor Scaling, Energy & Manufacturing Use Cases, and More New Articles on ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...