Deployment Architecture

Migration instructions from single install to distributed?

robertlynch2020
Influencer

Hi
I have a single install (Everything on one machine).

I want to go to one search head and 2 indexers (non clustered) multiple machines.

Is there a set of instruction on how to do this, the doc is great but there seem to be so many options that I get lost.
I am looking for step 1 2, 3..etc...

I also have a question like:
If I change the current install from SH+Indexer -> indexer and create a separate search head(I think this is the best way),
do I have to reinstall all my apps onto the new search head?

Regards
Robert

1 Solution

jnudell_2
Builder

Hi Robert,

There are plenty of Splunk documents on how to setup a distributed configuration (which you've probably encountered already):
New install: https://docs.splunk.com/Documentation/Splunk/7.2.6/DistSearch/Overviewofconfiguration

However, Google isn't really providing insight when migrating from a standalone to a distributed environment.

I think the easiest path would be as follows:
1. Install new Splunk instance (this will be the search head)
Linux: https://docs.splunk.com/Documentation/Splunk/7.3.0/Installation/InstallonLinux
Windows: https://docs.splunk.com/Documentation/Splunk/7.3.0/Installation/ChoosetheuserSplunkshouldrunas
2. Configure it to send data to the old instance/indexer-to-be
https://docs.splunk.com/Documentation/Splunk/7.2.6/DistSearch/Forwardsearchheaddata
3. Configure it to use the old instance/indexer-to-be as a search-peer (same thing as indexer, different terminology)
https://docs.splunk.com/Documentation/Splunk/7.2.6/DistSearch/Configuredistributedsearch
4. Copy your apps from the old instance to the new search head (/opt/splunk/etc/apps)
App migration reference: https://docs.splunk.com/Documentation/Splunk/latest/DistSearch/Migratefromstandalonesearchheads
6. Restart your search head
7. Review .conf files in /opt/splunk/etc/system/local to determine what needs to be moved to the search head

That's a rough overview, but should get you to where you want to be.

View solution in original post

richgalloway
SplunkTrust
SplunkTrust

If it was as easy as 1,2,3 the docs would say so. The docs have a lot of options because there are a lot of variables.

If I was in your shoes, I'd make the existing server an indexer and add new servers to act as search head and second indexer. Usually, it's not necessary to re-install apps - just transfer $SPLUNK_HOME/etc/apps from the old server to the new one. There are caveats so read the docs.
The second indexer will start out empty, but will accumulate data over time. Until then, however, searches won't benefit from the second indexer. Better is to cluster the indexes and balance the indexes from the start, but that's something for Professional Services to handle for you.

---
If this reply helps you, Karma would be appreciated.

robertlynch2020
Influencer

Hi

Thanks for this answer.
We have done the following. One new search head and 2 indexers. The 1st Indexer is the old production, but when we start it up all the datamodels start to rebuild. Is there a way to get the datamodels not to rebuild?

Thanks
Robert

0 Karma

jnudell_2
Builder

Hi Robert,

There are plenty of Splunk documents on how to setup a distributed configuration (which you've probably encountered already):
New install: https://docs.splunk.com/Documentation/Splunk/7.2.6/DistSearch/Overviewofconfiguration

However, Google isn't really providing insight when migrating from a standalone to a distributed environment.

I think the easiest path would be as follows:
1. Install new Splunk instance (this will be the search head)
Linux: https://docs.splunk.com/Documentation/Splunk/7.3.0/Installation/InstallonLinux
Windows: https://docs.splunk.com/Documentation/Splunk/7.3.0/Installation/ChoosetheuserSplunkshouldrunas
2. Configure it to send data to the old instance/indexer-to-be
https://docs.splunk.com/Documentation/Splunk/7.2.6/DistSearch/Forwardsearchheaddata
3. Configure it to use the old instance/indexer-to-be as a search-peer (same thing as indexer, different terminology)
https://docs.splunk.com/Documentation/Splunk/7.2.6/DistSearch/Configuredistributedsearch
4. Copy your apps from the old instance to the new search head (/opt/splunk/etc/apps)
App migration reference: https://docs.splunk.com/Documentation/Splunk/latest/DistSearch/Migratefromstandalonesearchheads
6. Restart your search head
7. Review .conf files in /opt/splunk/etc/system/local to determine what needs to be moved to the search head

That's a rough overview, but should get you to where you want to be.

woodcock
Esteemed Legend

Do not use Windows for your infrastructure here.

0 Karma

robertlynch2020
Influencer

noted we are on UNIX for everything

woodcock
Esteemed Legend

You have saved yourself much avoidable pain.

0 Karma

robertlynch2020
Influencer

Hi

Thanks for this, i will try it and get back

0 Karma

robertlynch2020
Influencer

Hi

Thanks for this answer.
We have done the following. One new search head and 2 indexers. The 1st Indexer is the old production, but when we start it up all the datamodels start to rebuild. Is there a way to get the datamodels not to rebuild?

Thanks
Robert

0 Karma

woodcock
Esteemed Legend

If you are going to use them, they must be rebuilt.

0 Karma

robertlynch2020
Influencer

Ok thanks - that is disappointing to here, it looks like the migration process will take ~48 hour of MAX CPU on a 60 core machine.

We have ~20 Data models.

Lucky we are on a very very good machine, otherwise we would have to stop production or do a parallel run

0 Karma
Get Updates on the Splunk Community!

How to Monitor Google Kubernetes Engine (GKE)

We’ve looked at how to integrate Kubernetes environments with Splunk Observability Cloud, but what about ...

Index This | How can you make 45 using only 4?

October 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this ...

Splunk Education Goes to Washington | Splunk GovSummit 2024

If you’re in the Washington, D.C. area, this is your opportunity to take your career and Splunk skills to the ...