Getting Data In

How to delete data from an index within a index cluster using SmartStore?

Jamie
Path Finder

Hello.  I am running 8.2.2 on Linux.  We have four clustered indexers and are using SmartStore.  I would like to empty an index (and recover the disk space).  I have thus chosen to remove the old_data index from the cluster, then add it back again.  I have performed these steps:

1. Stop any data being sent to the index.
2. Edit indexes.conf and delete the index's stanza (via the CM) then apply the changes to the peer nodes (each restarts).
3. Remove the index's directories from each peer node.
4. Check on the SHC for events in the index (index=old_data); no events are returned (all time).
5. Once the cluster shows that all indexes are 'green', re-add the index as normnal (editing indexes.conf again and applying the update).

However, now searching the index on the SHC returns some/most of the events.  My guess is that the cache manager / the S3 storage also needs to be purged.   If so, how is this best achieved?

I have avoided using index=old_data | delete because I understand this will only mask the data from searches (and I want the disk space back too).

Many thanks for your time.

Labels (3)
0 Karma
1 Solution

gcusello
SplunkTrust
SplunkTrust

Hi @Jamie,

put = 0 the retention for the index the you want to clean, setting  on the Master Node in the related stanza of indexes.conf:

FrozenTimePeriodInSecs = 0

then save and push the configurations to the indexers.

after few minutes, when the index is cleaned, you can set again the retention to the correct value.

Ciao.

Giuseppe

View solution in original post

gcusello
SplunkTrust
SplunkTrust

Hi @Jamie,

put = 0 the retention for the index the you want to clean, setting  on the Master Node in the related stanza of indexes.conf:

FrozenTimePeriodInSecs = 0

then save and push the configurations to the indexers.

after few minutes, when the index is cleaned, you can set again the retention to the correct value.

Ciao.

Giuseppe

Jamie
Path Finder

Ciao @gcusello,

Thank you for getting back to me.

Success!

Initially this did not work; the events continued to be returned from a search (I did wait 30 mins).  However, I had taken a tarball of the old_data directory on each indexer (plus old_data.dat) before starting.  So I:

- once again removed the old_data index from the cluster (i.e. updated indexes.conf from the CM).

- restored the tarball on each indexer.

- re-added the index back to indexes.conf.

- searched the data and saw the events as normal.

- edited indexes.conf setting FrozenTimePeriodInSecs = 0 for the old_data index.

However, I still saw the data with a search (but perhaps I should have waited longer, I beleive I waited 10+ minutes).  So I then changed FrozenTimePeriodInSecs = 1.  Perhaps a coincidence, but finally, the search returned no events.


Grazie!

Jamie.

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @Jamie,

good for you, see next time!

Ciao and happy splunking.

Giuseppe

P.S.: Karma Points are appreciated 😉

0 Karma
Get Updates on the Splunk Community!

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...