Splunk Enterprise

Splunk Clustered Indexer data erasing

smithke
Explorer

We have three indexers in our cluster.

We want to decommission one of the indexers but still want to search all the data on it. Therefore we are moving the old historical data from the indexer we want to decommission to another indexer in the cluster.

We have migrated all indexes over to the other indexer, however, there are about 5 indexes that as soon as we copy the data over to the other indexer it gets wiped from that indexer. Again we copy the data from the indexer we want to decommission and again on the indexer we want to keep it is nearly immediately removed.

I am guessing this is a cluster setting that is removing it. What am I missing? We only have to get these remaining indexes off so we can decommission this indexer.

Let me know if you need more clarification on the issue. I also have opened a ticket with Splunk support on the issue and will post the resolution here if support finds the issue before the community does.

 

Labels (3)
Tags (1)
1 Solution

richgalloway
SplunkTrust
SplunkTrust

You're trying too hard.  Just shut down the indexer with splunk offline --enforce-counts and let the Cluster Manager handle moving the data.

Also, make sure the cluster's RF/SF settings are consistent with a 2-node cluster.

---
If this reply helps you, Karma would be appreciated.

View solution in original post

richgalloway
SplunkTrust
SplunkTrust

You're trying too hard.  Just shut down the indexer with splunk offline --enforce-counts and let the Cluster Manager handle moving the data.

Also, make sure the cluster's RF/SF settings are consistent with a 2-node cluster.

---
If this reply helps you, Karma would be appreciated.

smithke
Explorer

Will this copy over all data so that we ensure we dont lose anything?

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Yes.  The --enforce-counts option tells the CM to make sure the RF and SF counts are satisfied before the indexer shuts down.

See https://docs.splunk.com/Documentation/Splunk/8.2.0/Indexer/Takeapeeroffline#Take_a_peer_down_permane... and https://docs.splunk.com/Documentation/Splunk/8.2.0/Indexer/Whathappenswhenapeergoesdown#When_a_peer_... .

---
If this reply helps you, Karma would be appreciated.
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...