How to delete data / index (reset start from scratch)

[mldeschenes]

From UI it seems easy to add data but I don't see an option to delete existing data from index. I need the quick an dirty steps to remove existing data/index so we can experiment different types of logs to import/indexed. Trial/Error pilot...

[daymauler]

This is how I took care of a similar issue:

1. I ran a search on the SH:

    index=my_index sourcetype=my_sourcetype | delete

2. After confirming that the records have been delete, I went to the HF

- stopped Splunk

-  /opt/splunk/bin/splunk cmd btprobe -d /opt/splunk/var/lib/splunk/fishbucket/splunk_private_db/ --file /my_file_path/filename.fileextension --reset

- restarted Splunk

and was able to see new data in SH

[Aleksey_18]

Prompt, how to clean an index when indexes in a cluster ?
The architecture of the cluster consists of two server indexers - the server indexer-1 and the server indexer-2, are managed all through the server cluster-master.
How in this case are the indexes cleaned correctly ?

In the documentation, I found 2 ways to clean the index:
1 Through the GUI https://docs.splunk.com/Documentation/Splunk/7.1.1/SearchReference/
2 And through the clean eventdata command http://docs.splunk.com/Documentation/Splunk/7.1.1/Indexer/RemovedatafromSplunk

The first option does not delete the physical data from the index, but makes them unavailable for searching
The second option cleans the index but completely, but in the documentation it is written that the command (clean eventdata) does NOT WORK FOR INDEXER IN THE CLUSTER
How will the PHYSICALLY index in the cluster be cleared ?

[goelli]

If you just want to wipe all data out of the index, perhaps you can use this:

Edit indexes.conf parameter frozenTimePeriodInSecs to 60 seconds
Apply cluster-bundle or restart indexer

Revert changes to indexes.conf and apply cluster-bundle or restart indexer again.

Best regards,
Thomas

[isoutamo]

And before that stop UFs and other inputs. I prefer even shorter period than 60 seconds. Wait until you see that this index is empty on all indexers, then update retention back.

Another option is remove index definition from indexes.conf apply cluster-bundle then go to indexers and remove individual index directories an files. Of course you must wait that rolling restart has finished before that.

Ismo

[lguinn2]

Great idea!

There isn't any great way to do this from the UI. However, there is a really simple way to do this from the command line:

splunk stop
splunk clean eventdata -index yourindex
splunk start

It cleans very fast, since it is just deleting all the files. If you want to clean ALL indexes, just leave off the

-index yourindex

[AnilPujar]

What is the steps for indexers in cluster?

[lguinn2]

If you still see previous data, then something didn't work. I use the clean command every week and it works, I assure you.

That said - if you reset the index, then IF you have an input defined in inputs.conf, the data will be re-indexed. If you don't want the data re-indexed, then disable the stanza in inputs.conf!

[mldeschenes]

So there is no way to reset the index , I still see previous log/data imported into index. I only had 1 log imported and basically want to start from scratch for testing purposes (test import options). I guess I will have to re-install the application every time it seems?

[miteshvohra]

It will be better to clean the _thefishbucket also to reindex the same data again.

[lguinn2]

If you clean an index, the fishbucket on the indexer will be appropriately updated so that the data will be reindexed - if that data came from the indexer.

However, if the data came from a forwarder, then you need to clean the _fishbucket on the forwarder to force the forwarder to resend data that it already sent once. (Because the forwarder doesn't know that you cleaned the indexer...)

So wherever the inputs.conf lives, that's where you need to reset the fishbucket to get the data re-indexed. If you don't want to reset the entire fishbucket, lookup btprobe which will let you reindex a single source.

Good point Mitesh

[lguinn2]

If you did this on the indexer, then all the indexes were cleaned. If you have multiple indexers, you will need to do this on each indexer.

splunklogger is used by Splunk for internal info - it will never "clean" and that's okay.

[mldeschenes]

Thank you for quick response, however did what you recommended without -index...
I still see Events Index data from UI?

splunk stop
splunk clean eventdata
This action will permanently erase all events from ALL indexes; it cannot be und
one.
Are you sure you want to continue [y/n]? y
....
Disabled database 'splunklogger': will not clean.
splunk start
All preliminary checks passed.
Done

[linu1988]

moreover |delete to hide the existing events and continue testing your data with different source config.

[rakesh_498115]

hi lguinn,

a quick one,

Can we clean the index without the flag , is there any option ??

thanks.