Splunk Dev

How to quickly remove indexes data from an indexer cluster

xsstest
Communicator

I have an indexer cluster with 8 indexers and a master node.

Now, I need to remove an index data, the index name is "tomcat"

How to quickly remove index data from an indexer cluster? distribute bundle?

only one by one delete indexs data from each indexer??

Tags (1)
0 Karma
1 Solution

DalJeanis
SplunkTrust
SplunkTrust

Seems like you could set that index to have a different retention schedule so the data would roll to cold and frozen quickly... assuming that it was being indexed into separate buckets, which would depend on configuration and whether you had originally set it up with any differences that would make the system think it ought to keep that index data separate.

Otherwise, you might be stuck with re-indexing everything in order to de-aggregate the lump of underlying data.

View solution in original post

0 Karma

DalJeanis
SplunkTrust
SplunkTrust

Seems like you could set that index to have a different retention schedule so the data would roll to cold and frozen quickly... assuming that it was being indexed into separate buckets, which would depend on configuration and whether you had originally set it up with any differences that would make the system think it ought to keep that index data separate.

Otherwise, you might be stuck with re-indexing everything in order to de-aggregate the lump of underlying data.

0 Karma

adonio
Ultra Champion

I believe @DalJeanis comment is a valid answer (not a comment), make sure there is no frozen path.
hard limit either the size for something like 10gb or the time for lets say last hour then distribute to indexers in the cluster.
all the data will roll very quickly out. all done.
another way is -> put cluster in maintenance mode -> stop indexers -> remove data with clean eventdata -index tomcat start all indexers -> disable maintenance mode.
if you will do it one indexer at a time, you will not suffer down time
hope it helps

0 Karma

neltonk
Path Finder

New to splunk please help...

I have a splunk cluster - 1 Master(also the license master), 3 node indexer cluster, 1 search head. I want to delete data in a specifi index

Could you please verify if the following steps are correct to delete event data?

On the Master Node : put cluster in maintenance mode
stop indexers - splunk stop on each indexer
remove data using the command splunk clean eventdate -index xyz - where do I run this command - on each indexer node ?
start indexers - splunk start on each indexer
On Master Node : disable Maintenance node

0 Karma

dineshraj9
Builder

You can use the | delete command to make the data inaccessible to the users. This won't free up space on the indexers though.

http://docs.splunk.com/Documentation/Splunk/6.6.0/SearchReference/Delete

For this you need a user with can_delete role(if you already don't have then create a temporary user and assign this role) and then run the query to return all data and delete. index=tomcat earliest=0 | delete and delete the data.

To permanently remove event data from a single index, type:
$SPLUNK_HOME/bin/splunk clean eventdata -index <index_name>

Details are provided here - http://docs.splunk.com/Documentation/Splunk/6.6.1/Indexer/RemovedatafromSplunk

For freeing up space, remove any monitors sending data to this index, remove index configuration from indexes.conf, push bundle and delete the index directory from the indexers $SPLUNK_HOME/var/lib/splunk/tomcat.

0 Karma

xsstest
Communicator

The delete command does not apply to indexer clusters

0 Karma

xsstest
Communicator

And I want to free up disk space

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...