Splunk Enterprise (Windows): move to new hardware and then upgrade

[Marco-IT]

Hello everybody,

I am upgrading Splunk Enterprise from 7.3.X to 8.2.5 (Windows). Due to the compatibility, I also need a more recent Windows version on my hosts to support Splunk. Therefore, I'm gonna use a new host for each server.

The architecture includes:
- 1 cluster master
- 1 deployment servers
- 1 search head
- 2 indexers (cluster)
- 1 poller (heavy forwarder)
- n universal forwarders

I've found HERE how to migrate a Splunk Enterprise instance from one physical machine to another, can anybody confirm me the following procedure?

- Stop Splunk Enterprise services on the host from which I want to migrate
- Roll any hot buckets on the source host from hot to warm
- Copy the entire contents of the $SPLUNK_HOME directory and all the directories containing buckets from the old host to the new one
- Turn off the old host
- Configure the new host in order to have the same IP address and hostname of the old host. This avoid not to redirect forwarders to the new instance
- Install Splunk Enterprise 7.3.X on the new host
- Verify that the index configuration (indexes.conf) file's volume, sizing, and path settings are still valid on the new host.
- Start Splunk Enterprise on the new instance.
- Log into Splunk Enterprise and confirm that your data is intact by searching it
- Upgrade from 7.3.X to 8.1.X and then to 8.2.5

Should I apply these steps to every host? What about the two indexers? I'm gonna need to migrate data, what's the correct procedure?
Also, I'm afraid that the new installation would reingest data from the poller, should I do something to prevent it?
Last thing: I'm gonna probably need to change the IP of one indexer, when should I change its configurations?

Thanks in advance for any help.

[gcusello · Accepted solution]

Hi @Marco-IT,

yes, the procedure is correct, I'd change only one step: step 3 (Copy the entire contents of the $SPLUNK_HOME directory and all the directories containing buckets from the old host to the new one):

you don't need to copy all the $SPLUNK_HOME folder, you need only etc and var.

About the second question, yes, you have to apply this process to all your hosts but it's simple because you have only a little Indexer cluster.

Remember the priorities in migration:

• 1 cluster master
  
• 1 search head
• 2 indexers (cluster)
• 1 deployment servers
• 1 poller (heavy forwarder)
• universal forwarders

About the IP changing of an Indexer, it could be a problem and it should be better to avoid it, but if you need, maybe the easier approach is to add another node to the cluster, wait that the replication is complite, then cancel a node from the cluster.

You can find instruction ins Indexer cluster upgrade at https://docs.splunk.com/Documentation/Splunk/8.2.5/Indexer/Upgradeacluster

Ciao.

Giuseppe

[anem]

was looking for something like this , thanks for the info , do u have any doc of process or an article that was written , kindly share 

thank you 

[gcusello]

Hi @anem,

it's always better to open a new question than add a post to an existing one, so more people can help you better and quickly.

Anyway, you can find some interesting documentation at:

https://wiki.splunk.com/Deploy:Migrating_a_Splunk_Install

https://docs.splunk.com/Documentation/Splunk/8.2.5/Installation/MigrateaSplunkinstance

Ciao.

Giuseppe

[anem]

thank you so much

[Marco-IT]

Thank you @gcusello for your quick answer.

Why should I copy only etc and var if the official documentation says the whole $SPLUNK_HOME folder (I noticed other users said the same on other answers from other topics).

What about the buckets? they aren't in etc or var, shouldn't I copy them?

[gcusello]

Hi @Marco-IT,

in etc, you have all the configurations files, in var you have all the data (if you didn't changed the DB_HOME), log and run files; all the other folders contain executebles, libraries, in few words all things that you usually don't modify respect the installation, so you already have them when you install the old Splunk version.

Especially on Windows, it isn't relevant to copy executables files, if you are running Splunk on Linuc, you could copy the full $SPLUNK_HOME, on windows isn't relevant.

About buckets, by default they are in $SPLUNK_HOME\var\lib\splunk, obviously, if you have indexes (buckets) in a different folder, you have to copy all of them!

Only one final hint: put much attention to use Windows as production operative system: I didn't see large Splunk infrastructure on Windows, only little or test installations!

Also because, if you have to manage Linux Forwarders with a windows Deployment Server, you'll securely have problems!

Ciao.

Giuseppe