Splunk on new servers

[rahul2gupta]

Hi @gcusello ,

Background:

We tried to upgrade our existing environment(7.1.3) to higher version 8.1 but we were unable to do so because of some issue and failed multiple times to upgrade our Indexer and also we couldn't get much help from Splunk support.

Present scenario:

Instead of upgrading, we planned to install Splunk on new servers (Indexer & SH) and we were able to do so and luckily we also able to map Indexer and SH.
Mainly we intend to built everything in Splunk from scratch(Replica to our existing Splunk environment).

Next step I did was to find the hosts from where the indexes are getting data but its difficult to get all the indexes that are used by different apps(Number of Dashboard is high). Is there any query to get index/s that are being used by different Apps?

Also, can you please help me to guide how to achieve this(Steps)?

Regards,
Rahul

[isoutamo]

Hi

You can do this with next steps

1. Copy old data e.g. with rsync to new servers one by one. This can be do when all nodes are up
2. Install new splunk version to new servers
3. Stop old servers
4. Sync data + configs again from old ones to new ones (this must do with option: delete removed etc.)
5. Update configs if needed
6. Start new environment in correct order based on your environment 

When you are starting with new servers, you probably update also OS to newer and this can (usually) means that there are some new things to do. Fortunately you can update configs after step 2 and test everything with new names etc. After you have verified that everything is working you can change back splunk configurations or actually those will be overwritten when you are doing resync on step 4. 

One old answer for this https://community.splunk.com/t5/Installation/How-to-migrate-indexes-to-new-indexer-instance/m-p/5281...

With this process you don't need to do those app installations and configurations as you just copied those from old to new. 

Basically you cannot get 100% sure all indexes which are used by any SPL. There can be queries without any index=xyz definitions or those are defined on some other config files which are not expanded to audit log. There are some answers where you can get some ideas to get this list as well as it can be.

r. Ismo