Deployment Architecture

Migration instructions from single install to distributed?

Motivator

Hi
I have a single install (Everything on one machine).

I want to go to one search head and 2 indexers (non clustered) multiple machines.

Is there a set of instruction on how to do this, the doc is great but there seem to be so many options that I get lost.
I am looking for step 1 2, 3..etc...

I also have a question like:
If I change the current install from SH+Indexer -> indexer and create a separate search head(I think this is the best way),
do I have to reinstall all my apps onto the new search head?

Regards
Robert

1 Solution

Builder

Hi Robert,

There are plenty of Splunk documents on how to setup a distributed configuration (which you've probably encountered already):
New install: https://docs.splunk.com/Documentation/Splunk/7.2.6/DistSearch/Overviewofconfiguration

However, Google isn't really providing insight when migrating from a standalone to a distributed environment.

I think the easiest path would be as follows:
1. Install new Splunk instance (this will be the search head)
Linux: https://docs.splunk.com/Documentation/Splunk/7.3.0/Installation/InstallonLinux
Windows: https://docs.splunk.com/Documentation/Splunk/7.3.0/Installation/ChoosetheuserSplunkshouldrunas
2. Configure it to send data to the old instance/indexer-to-be
https://docs.splunk.com/Documentation/Splunk/7.2.6/DistSearch/Forwardsearchheaddata
3. Configure it to use the old instance/indexer-to-be as a search-peer (same thing as indexer, different terminology)
https://docs.splunk.com/Documentation/Splunk/7.2.6/DistSearch/Configuredistributedsearch
4. Copy your apps from the old instance to the new search head (/opt/splunk/etc/apps)
App migration reference: https://docs.splunk.com/Documentation/Splunk/latest/DistSearch/Migratefromstandalonesearchheads
6. Restart your search head
7. Review .conf files in /opt/splunk/etc/system/local to determine what needs to be moved to the search head

That's a rough overview, but should get you to where you want to be.

View solution in original post

SplunkTrust
SplunkTrust

If it was as easy as 1,2,3 the docs would say so. The docs have a lot of options because there are a lot of variables.

If I was in your shoes, I'd make the existing server an indexer and add new servers to act as search head and second indexer. Usually, it's not necessary to re-install apps - just transfer $SPLUNK_HOME/etc/apps from the old server to the new one. There are caveats so read the docs.
The second indexer will start out empty, but will accumulate data over time. Until then, however, searches won't benefit from the second indexer. Better is to cluster the indexes and balance the indexes from the start, but that's something for Professional Services to handle for you.

---
If this reply helps you, an upvote would be appreciated.

Motivator

Hi

Thanks for this answer.
We have done the following. One new search head and 2 indexers. The 1st Indexer is the old production, but when we start it up all the datamodels start to rebuild. Is there a way to get the datamodels not to rebuild?

Thanks
Robert

0 Karma

Builder

Hi Robert,

There are plenty of Splunk documents on how to setup a distributed configuration (which you've probably encountered already):
New install: https://docs.splunk.com/Documentation/Splunk/7.2.6/DistSearch/Overviewofconfiguration

However, Google isn't really providing insight when migrating from a standalone to a distributed environment.

I think the easiest path would be as follows:
1. Install new Splunk instance (this will be the search head)
Linux: https://docs.splunk.com/Documentation/Splunk/7.3.0/Installation/InstallonLinux
Windows: https://docs.splunk.com/Documentation/Splunk/7.3.0/Installation/ChoosetheuserSplunkshouldrunas
2. Configure it to send data to the old instance/indexer-to-be
https://docs.splunk.com/Documentation/Splunk/7.2.6/DistSearch/Forwardsearchheaddata
3. Configure it to use the old instance/indexer-to-be as a search-peer (same thing as indexer, different terminology)
https://docs.splunk.com/Documentation/Splunk/7.2.6/DistSearch/Configuredistributedsearch
4. Copy your apps from the old instance to the new search head (/opt/splunk/etc/apps)
App migration reference: https://docs.splunk.com/Documentation/Splunk/latest/DistSearch/Migratefromstandalonesearchheads
6. Restart your search head
7. Review .conf files in /opt/splunk/etc/system/local to determine what needs to be moved to the search head

That's a rough overview, but should get you to where you want to be.

View solution in original post

Esteemed Legend

Do not use Windows for your infrastructure here.

0 Karma

Motivator

noted we are on UNIX for everything

Esteemed Legend

You have saved yourself much avoidable pain.

0 Karma

Motivator

Hi

Thanks for this, i will try it and get back

0 Karma

Motivator

Hi

Thanks for this answer.
We have done the following. One new search head and 2 indexers. The 1st Indexer is the old production, but when we start it up all the datamodels start to rebuild. Is there a way to get the datamodels not to rebuild?

Thanks
Robert

0 Karma

Esteemed Legend

If you are going to use them, they must be rebuilt.

0 Karma

Motivator

Ok thanks - that is disappointing to here, it looks like the migration process will take ~48 hour of MAX CPU on a 60 core machine.

We have ~20 Data models.

Lucky we are on a very very good machine, otherwise we would have to stop production or do a parallel run

0 Karma