How to update indexes.conf files on unclustered production indexers?

[packet_hunter]

I have to define some new indexes on production indexers (in the indexes.conf).
I have 4 indexers running.
Someone else setup an app to send_data_to_indexers (a basic outputs.conf) as follows

[tcpout]
 defaultGroup = default-autolb-group

 [tcpout:default-autolb-group]

 server = splunkindexer1.mycorp.com:9997, splunkindexer2.mycorp.com:9997, splunkindexer3.mycorp.com:9997, splunkindexer4.mycorp.com:9997


 [tcpout-server://splunkindexer1.mycorp.com:9997]

My question is: If this outputs.conf is being used for all data being sent to the indexers, then can I edit the indexes.conf on each indexer and then restart one at a time?

Or is there a better way to do this?

Thank you

[gcusello · Accepted solution]

Hi packet_hunter,

in your outputs.conf you should have four lines as the last one

[tcpout-server://splunkindexer1.mycorp.com:9997]
[tcpout-server://splunkindexer2.mycorp.com:9997]
[tcpout-server://splunkindexer3.mycorp.com:9997]
[tcpout-server://splunkindexer4.mycorp.com:9997]

Anyway, I see that you configured your indexers in auto load balancing so, if one of them is down for update, the others continue to receive logs from Universal Forwarders.
The only problem is that, during downtime, data on this indexers aren't searchable.

What's your requirement: don't lose any log or have always logs searchable?

If your requirement is don't lose any log, you haven't problems; in addition remember that receiving logs only from Universal Forwarder you can also stop all the indexers at the same time, because UFs cache logs when Indexers aren't available.

If instead your requirement is to always have logs searchable, you must use an Indexer Cluster.

Bye.
Giuseppe

[packet_hunter]

Thank you Cusello.

I plan to update after hours, search-ability should not be a big concern. Primary concern is to not lose data.
I really appreciate your insight.

Regarding your code that you provided above, do I need to rewrite the code that I am using for autoLB?
If I am understanding correctly, is this what you mean?

[tcpout]
 defaultGroup = default-autolb-group

 [tcpout:default-autolb-group]

 server = splunkindexer1.mycorp.com:9997, splunkindexer2.mycorp.com:9997, splunkindexer3.mycorp.com:9997, splunkindexer4.mycorp.com:9997



 [tcpout-server://splunkindexer1.mycorp.com:9997]
 [tcpout-server://splunkindexer2.mycorp.com:9997]
 [tcpout-server://splunkindexer3.mycorp.com:9997]
 [tcpout-server://splunkindexer4.mycorp.com:9997]

[gcusello]

Hi packet_hunter,
Yes correct!

how do you deploy outputs.conf?
I suggest to insert it in a dedicated TA to deploy using Deployment Server.
In this way you have a more feasible solution: you can modify outputs.conf of all UFs in one shot.

Bye.
Giuseppe

[packet_hunter]

Thank you for confirming.
Yes we use the deployment server to push out the output.conf as an app to the UFs.

For example we create an app called OutputsToIndexers

and within this app is the code I provided above.

Is this scenario what you are recommending?

[gcusello]

Yes.
Thank You.
Bye.
Giuseppe