Getting Data In

Can I add the existing data to the cluster by adding the standalone instance to the cluster as a peer?

whar_garbl
Path Finder

I have a standalone instance with existing data on it. I have created a new indexer cluster that does not include this standalone machine. All instances are running the same OS and Splunk version.

Can I add the existing data to the cluster by adding the standalone instance to the cluster as a peer? What will the behavior be in such a case? 

I'm aware of the bucket copying method, but I'm hoping there's a more hands-off method to accomplish this goal. 

Labels (1)
0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @whar_garbl,

even if you add the standalone indexer to a cluster, the old data aren't replicated to the other Indexers, so there isn't any automatic solution for yur requirement.

If you want to have the old data in all indexers you have to use a porkaround like this:

  • use on clustered indexers an index with a different name than the old one,
  • stopping Splunk, copy the old indexes in all the clustered Indexers,
  • create an eventtype with both the indexes (the old and the new ones) and use it in all your searches.

In this way you have the old data in all indexers and you can use in your searches both the old and the new data.

The new data will be stored in the new indexes and the old ones will be empty in the future after the retention time.

I agree that it isn't a good solution, but I don't see anything else than don't replicate the old data (for this reason I defined this solution a "porkaround"!).

Ciao.

Giuseppe

whar_garbl
Path Finder

That's unfortunate. I have super long retention requirements, so I'd effectively be permanently keeping all the old and new data in their separate sources. I guess I'm going to go with the bucket copying hack. I don't like it, but I don't see a better way. 

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @whar_garbl,

as I said, even if you manually copy buckets in a clustered Indexer, these buckets will not be automatically replicated to the others, for this reason I hinted the porkaround.

I had the same problem some years ago and I implemented the descripted porkaround, that I can confirm that really works, but it could be long to implement because you have to modify all your searches (Alerts, Reports and Dashboard Panels) using an eventtype instead one index; to be ready for an issue like this I usually use eventtypes instead of index in my searches.

The only other way is manually reindex all your data, but it's a very hard work, it's easier the porkaround!

tell me if I can help you more, otherwise, please accept one answer for the other people of Community.

Ciao.

Giuseppe

P.S.: Karma Points are appreciated 😉

whar_garbl
Path Finder

Correct me if I'm wrong here - I believe the right way to do the hack I'm talking about is to break the new cluster (it has no data or function as of now), then copy the buckets from the Old Indexer to one of the New (and now standalone) Indexers, then re-instantiate the cluster, which will then replicate the buckets to all its new best friends. It's not trivial to reconfigure the cluster and its associated SHC, though. 

I'm trying to avoid doing the eventtypes 'porkaround' because as you say, it will break a ton of searches, and I know the people who rely on those searches are not good at SPL; it will likely fall to me to cobble it back together. That will be time-consuming, to say the least. 

Reindexing is also not an option - that sounds like a nightmare, from my cursory reading. 

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @whar_garbl,

I'm not sure that you can copy buckets from a non clustered Indexer to a clustered one: I think that you canno see the data and anyway, data will not be replicated.

As I said, you have only two choices: the porkaround or extract and reindex all data and I don't know which is the worst!

There's a third solution but it's very expensive: call Splunk Professional Services, they have know how and tools to do this.

Ciao.

Giuseppe

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...