Deployment Architecture

How to extend restart period for single indexer that I wish to take offline

ezmo1982
Path Finder

Hi 

I have a single indexer in my splunk on-prem environment. I wish to take the Indexer offline so that I can perform a system upgrade. I am running on RHEL 8.3.

The docs state the default restart period after issuing the splunk offline command is 60 seconds. I wish to change this to 60 minutes so that the upgrade will complete in time, before issuing the splunk restart command. The docs state that i can do this by issuing the command, passing 3600 as the argument.

splunk edit cluster-config -restart_timeout <seconds>

 However when i run this command on my indexer i get the following message:

mode=disabled cannot edit config. Please pick a mode [master|slave|searchhead] to edit clustering properties

Which option should i use? Im reluctant to use any of the options as I dont have clustering enabled in my environment (no clustering  for Indexing or Search heads) so dont want to make any unnecessary changes.

Thanks.

Labels (1)
0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

The docs are a little misleading or easily misread.

The offline command does not automatically restart the indexer after any period of time.  What happens after 60 seconds is the cluster resumes bucket fix-up  The stopped indexer still must be started using the splunk restart command.  The restart_timeout setting refers to how long the cluster will wait before restarting bucket fixup rather than restarting the indexer.

None of that applies to unclustered indexers.  For a single indexer, just issue splunk stop, perform the upgrade, then issue splunk start.  Be sure to perform a backup first, just in case.  Also, be aware that forwarders will cache data while the indexer is down, but data may be lost if the outage is prolonged.  Any data arriving on TCP or UDP ports while the indexer is down is guaranteed to be lost.

---
If this reply helps you, an upvote would be appreciated.

View solution in original post

richgalloway
SplunkTrust
SplunkTrust

The docs are a little misleading or easily misread.

The offline command does not automatically restart the indexer after any period of time.  What happens after 60 seconds is the cluster resumes bucket fix-up  The stopped indexer still must be started using the splunk restart command.  The restart_timeout setting refers to how long the cluster will wait before restarting bucket fixup rather than restarting the indexer.

None of that applies to unclustered indexers.  For a single indexer, just issue splunk stop, perform the upgrade, then issue splunk start.  Be sure to perform a backup first, just in case.  Also, be aware that forwarders will cache data while the indexer is down, but data may be lost if the outage is prolonged.  Any data arriving on TCP or UDP ports while the indexer is down is guaranteed to be lost.

---
If this reply helps you, an upvote would be appreciated.

ezmo1982
Path Finder

The docs state not to use splunk stop (per below).  Does this just apply to clusters?

Caution: Do not use splunk stop to take a peer offline. Instead, use splunk offline. It stops the peer in a way that minimizes disruption to your searches.

 

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Yes, since that quotation is in the Manage the Indexer Cluster section of the document, it only applies to clusters.

---
If this reply helps you, an upvote would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Maximize the Value from Microsoft Defender with Splunk

 Watch NowJoin Splunk and Sens Consulting for this Security Edition Tech TalkWho should attend:  Security ...

This Week's Community Digest - Splunk Community Happenings [6.27.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...

Reminder! Splunk Love Promo: $25 Visa Gift Card for Your Honest SOAR Review With ...

We recently launched our first Splunk Love Special, and it's gone phenomenally well, so we're doing it again, ...