topic Re: How to delete indexed data from disk? in Getting Data In

How to delete indexed data from disk?

vtrend — Wed, 11 Aug 2021 23:33:04 GMT

I have an index = 'telemetry' which gets data from a local directory on standalone Splunk installation.

I deleted some data from above index which came in from particular directory using command

index='telemetry' source="/data/01/*" | delete

The above index has still more data from other sources (e.g. "/data/02" ..)

I want to re-index the data from deleted directory i.e. "/data/01" again.
Running splunk clean eventdata involves deleting entire index.

I want to wipe from disk only that part of data that has been deleted above so that I can re-index it back. How can I achieve this?

Re: How to delete indexed data from disk?

venkatasri — Thu, 12 Aug 2021 00:11:01 GMT

Hi @vtrend

There is no other command except clean to wipe out data from disk however as you said it will do whole index not a particular source.

| delete command is something you already did meaning you won't be able to search it again but data exist on disk. When you re-ingest data from same source that you have deleted newly ingested will be searchable without issues.

Hope this helps!

Re: How to delete indexed data from disk?

vtrend — Thu, 12 Aug 2021 00:39:00 GMT

@venkatasri Thanks Venkat for helping me out.

I tested by re-ingesting same data but Splunk is not indexing it back, the job runs, but no results. Searching around the forums, few users mentioned I need to either clean the index or delete fishbuckets etc which I was trying to avoid.

Re: How to delete indexed data from disk?

venkatasri — Thu, 12 Aug 2021 00:42:09 GMT

@vtrend

How are you re-ingesting the data from same source?

Re: How to delete indexed data from disk?

vtrend — Thu, 12 Aug 2021 00:46:15 GMT

Yes, I have added local data directory with the path "/data/01/" and "Indexed Once' again.
I chose same Index and App Context while re-ingesting.

However when I submit and complete above steps, the search query gives 0 results.

index='telemetry' source='/data/01/*'

Re: How to delete indexed data from disk?

venkatasri — Thu, 12 Aug 2021 00:49:56 GMT

@vtrend If you are using the same forwarder to re-index fishbucket ignores re-reading the same file contents.

Try clearing the fishbucket to specific source, replace - "/var/log/messages" with your source path

./splunk cmd btprobe -d /opt/splunkforwarder/var/lib/splunk/fishbucket/splunk_private_db --file /var/log/messages --reset

Re: How to delete indexed data from disk?

vtrend — Thu, 12 Aug 2021 00:54:58 GMT

@venkatasri

Ohh, this explains it. I'll try this and update here with the results.

Thanks a lot.