Getting Data In

Shutting down splunk Indexers For Upgrade

paul_1994
Path Finder

I am in a situation where I need to migrate my the splunk indexes to a bigger drive. I was wondering what would be a good way of accomplishing this.

I guess my question is what is the best way to shutdown these servers and upgrade them one at a time? Will this cause any issues? What happens with The Universal Forwarders?

My environment consists of 2 Search heads and 2 indexers with several Universal forwarders sending logs.

  1. my concern is what happens when I shutdown an Indexer.Does all the new data just go to one Indexer?
  2. When upgrading each server is there a problem having this Server down for 2-3 hours?
0 Karma

gkanapathy
Splunk Employee
Splunk Employee

Queue whatever the forwarder would hold, which is normally only a few hundred or thousand events, then the forwarders would stop accepting data.

paul_1994
Path Finder

What would happen if both indexers were down. Does the data just queue? Or does it gets lost and I will miss all data coming in from the UF's? If so what is the best way to make sure I don't lose any new data?

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

If you are sending from Splunk forwarders, then yes, while one indexer is down, all new data will go to the remaining ones. Assuming that one indexer can handle the load, the downside of this is really just that your data for that period will be unbalanced, so if you search for any data collected during the time, it's all stored on one node, so one node does all the work of retrieving the data. Over time, both will balance out, and if it's for a relatively short period (a few hours) there is no long-term harm. Of course the other disadvantages would be that if your one remaining server stopped while you were upgrading, you would of course be unable to index at all, and during the upgrade, data on the down indexer will be unavailable (and so searches will return incomplete results) but that I think is and obvious consequence.

paul_1994
Path Finder

What would happen if both indexers were down. Does the data just queue? Or does it gets lost and I will miss all data coming in from the UF's? If so what is the best way to make sure I don't lose any new data?

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Modernize your Splunk Apps – Introducing Python 3.13 in Splunk

We are excited to announce that the upcoming releases of Splunk Enterprise 10.2.x and Splunk Cloud Platform ...

Step into “Hunt the Insider: An Splunk ES Premier Mystery” to catch a cybercriminal ...

After a whole week of being on call, you fell asleep on your keyboard, and you hit a sequence of buttons that ...

SplunkTrust Application Period is Officially OPEN!

It's that time, folks! The application/nomination period for the 2026-2027 SplunkTrust is officially open. If ...