topic Re: We need to upgrade the hardware on 1 of our Indexer server. What is the best approach for downtime for this upgrade? in Monitoring Splunk

We need to upgrade the hardware on 1 of our Indexer server. What is the best approach for downtime for this upgrade?

Gh0st_rid3r — Mon, 22 Nov 2021 13:10:56 GMT

Our Splunk Indexer is under resourced. To match Splunk support's recommendations we need to add more RAM to it. We have a deployment server with 2 indexer & 2 search head. This upgrade will require about 30 minutes of downtime .
What's the best approach for the hardware upgrade?

Re: We need to upgrade the hardware on 1 of our Indexer server. What is the best approach for downtime for this upgrade?

aasabatini — Mon, 22 Nov 2021 13:56:13 GMT

HI @Gh0st_rid3r

Put the cluster in maintenance mode, stop the cluster master, shut down the cluster master, perform the upgrade, restart the cluster master, restart splunk (note splunk may be set to auto start), exiting maintenance mode occurs automatically with restarts.

Maintenance mode does not persist across master restarts.

http://docs.splunk.com/Documentation/Splunk/6.5.2/Indexer/Usemaintenancemode

Re: We need to upgrade the hardware on 1 of our Indexer server. What is the best approach for downtime for this upgrade?

richgalloway — Mon, 22 Nov 2021 13:57:49 GMT

Based on the description of your environment, I'm presuming the indexers are not clustered.

1) Ensure all forwarders are load-balancing between the two indexers.

2) Stop the indexer.

3) Add RAM

4) Restart the indexer

While the indexer is down data will continue to be sent to the remaining indexer so no data should be lost. Searches, however, will return errors about being unable to reach all indexers and possibly having incomplete results. For that reason, you may want to plan the outage around any important scheduled reports.

Re: We need to upgrade the hardware on 1 of our Indexer server. What is the best approach for downtime for this upgrade?

Gh0st_rid3r — Mon, 22 Nov 2021 14:00:02 GMT

@aasabatini wrote:
HI @Gh0st_rid3r

Put the cluster in maintenance mode, stop the cluster master, shut down the cluster master, perform the upgrade, restart the cluster master, restart splunk (note splunk may be set to auto start), exiting maintenance mode occurs automatically with restarts.
Maintenance mode does not persist across master restarts.
http://docs.splunk.com/Documentation/Splunk/6.5.2/Indexer/Usemaintenancemode

Hi @aasabatini , But that applies to the cluster master setup. In our environment, We dont have a cluster master. Just the primary & backup indexers as same as search head. A deployment server for the configuration as per my knowledge. But thanks anyway.

Re: We need to upgrade the hardware on 1 of our Indexer server. What is the best approach for downtime for this upgrade?

aasabatini — Mon, 22 Nov 2021 14:02:30 GMT

Hi @Gh0st_rid3r

sorry I supposed you have a cluster enviroment. anyway I'm happy you resolved your doubs.

Regards

Ale

Re: We need to upgrade the hardware on 1 of our Indexer server. What is the best approach for downtime for this upgrade?

Gh0st_rid3r — Mon, 22 Nov 2021 14:03:58 GMT

Thanks @richgalloway . You are right. The indexers are not clustered as per my knowledge. Havent come across the cluster master in our setup. Just indexers,search heads & deployment server.

Thanks. Will follow the steps as you suggested 😀.

Re: We need to upgrade the hardware on 1 of our Indexer server. What is the best approach for downtime for this upgrade?

Gh0st_rid3r — Mon, 22 Nov 2021 14:04:30 GMT

Thanks mate.