topic Re: How to delete data / index (reset start from scratch) in Getting Data In

How to delete data / index (reset start from scratch)

mldeschenes — Tue, 06 May 2014 17:25:13 GMT

From UI it seems easy to add data but I don't see an option to delete existing data from index. I need the quick an dirty steps to remove existing data/index so we can experiment different types of logs to import/indexed. Trial/Error pilot...

Re: How to delete data / index (reset start from scratch)

lguinn2 — Tue, 06 May 2014 17:37:37 GMT

Great idea!

There isn't any great way to do this from the UI. However, there is a really simple way to do this from the command line:

splunk stop
splunk clean eventdata -index yourindex
splunk start

It cleans very fast, since it is just deleting all the files. If you want to clean ALL indexes, just leave off the

-index yourindex

Re: How to delete data / index (reset start from scratch)

linu1988 — Tue, 06 May 2014 17:45:17 GMT

moreover |delete to hide the existing events and continue testing your data with different source config.

Re: How to delete data / index (reset start from scratch)

mldeschenes — Tue, 06 May 2014 17:58:24 GMT

Thank you for quick response, however did what you recommended without -index...
I still see Events Index data from UI?

splunk stop splunk clean eventdata This action will permanently erase all events from ALL indexes; it cannot be und one. Are you sure you want to continue [y/n]? y .... Disabled database 'splunklogger': will not clean. splunk start All preliminary checks passed. Done

Re: How to delete data / index (reset start from scratch)

lguinn2 — Tue, 06 May 2014 23:18:36 GMT

If you did this on the indexer, then all the indexes were cleaned. If you have multiple indexers, you will need to do this on each indexer.

splunklogger is used by Splunk for internal info - it will never "clean" and that's okay.

Re: How to delete data / index (reset start from scratch)

miteshvohra — Wed, 07 May 2014 02:23:34 GMT

It will be better to clean the _thefishbucket also to reindex the same data again.

Re: How to delete data / index (reset start from scratch)

mldeschenes — Wed, 07 May 2014 14:58:58 GMT

So there is no way to reset the index , I still see previous log/data imported into index. I only had 1 log imported and basically want to start from scratch for testing purposes (test import options). I guess I will have to re-install the application every time it seems?

Re: How to delete data / index (reset start from scratch)

lguinn2 — Thu, 08 May 2014 03:55:10 GMT

If you still see previous data, then something didn't work. I use the clean command every week and it works, I assure you.

That said - if you reset the index, then IF you have an input defined in inputs.conf, the data will be re-indexed. If you don't want the data re-indexed, then disable the stanza in inputs.conf!

Re: How to delete data / index (reset start from scratch)

lguinn2 — Thu, 10 Sep 2015 23:39:03 GMT

If you clean an index, the fishbucket on the indexer will be appropriately updated so that the data will be reindexed - if that data came from the indexer.

However, if the data came from a forwarder, then you need to clean the _fishbucket on the forwarder to force the forwarder to resend data that it already sent once. (Because the forwarder doesn't know that you cleaned the indexer...)

So wherever the inputs.conf lives, that's where you need to reset the fishbucket to get the data re-indexed. If you don't want to reset the entire fishbucket, lookup btprobe which will let you reindex a single source.

Good point Mitesh

Re: How to delete data / index (reset start from scratch)

goelli — Mon, 14 Dec 2015 10:45:47 GMT

If you just want to wipe all data out of the index, perhaps you can use this:

Edit indexes.conf parameter frozenTimePeriodInSecs to 60 seconds
Apply cluster-bundle or restart indexer

Revert changes to indexes.conf and apply cluster-bundle or restart indexer again.

Best regards,
Thomas

Re: How to delete data / index (reset start from scratch)

rakesh_498115 — Tue, 22 Dec 2015 13:28:16 GMT

hi lguinn,

a quick one,

Can we clean the index without the flag , is there any option ??

thanks.

Re: How to delete data / index (reset start from scratch)

Aleksey_18 — Wed, 27 Jun 2018 14:04:02 GMT

Prompt, how to clean an index when indexes in a cluster ?
The architecture of the cluster consists of two server indexers - the server indexer-1 and the server indexer-2, are managed all through the server cluster-master.
How in this case are the indexes cleaned correctly ?

In the documentation, I found 2 ways to clean the index:
1 Through the GUI https://docs.splunk.com/Documentation/Splunk/7.1.1/SearchReference/
2 And through the clean eventdata command http://docs.splunk.com/Documentation/Splunk/7.1.1/Indexer/RemovedatafromSplunk

The first option does not delete the physical data from the index, but makes them unavailable for searching
The second option cleans the index but completely, but in the documentation it is written that the command (clean eventdata) does NOT WORK FOR INDEXER IN THE CLUSTER
How will the PHYSICALLY index in the cluster be cleared ?

Re: How to delete data / index (reset start from scratch)

AnilPujar — Wed, 06 Feb 2019 07:25:24 GMT

What is the steps for indexers in cluster?

Re: How to delete data / index (reset start from scratch)

isoutamo — Fri, 28 Feb 2020 06:30:53 GMT

And before that stop UFs and other inputs. I prefer even shorter period than 60 seconds. Wait until you see that this index is empty on all indexers, then update retention back.

Another option is remove index definition from indexes.conf apply cluster-bundle then go to indexers and remove individual index directories an files. Of course you must wait that rolling restart has finished before that.

Ismo

Re: How to delete data / index (reset start from scratch)

daymauler — Fri, 04 Jun 2021 13:35:29 GMT

This is how I took care of a similar issue:

1. I ran a search on the SH:

index=my_index sourcetype=my_sourcetype | delete

2. After confirming that the records have been delete, I went to the HF

- stopped Splunk

- /opt/splunk/bin/splunk cmd btprobe -d /opt/splunk/var/lib/splunk/fishbucket/splunk_private_db/ --file /my_file_path/filename.fileextension --reset

- restarted Splunk

and was able to see new data in SH