What are the pros and cons of using Indexer Discovery?

[brent_weaver]

Hello all... Just writing to see what experiences people are having with Indexer Discovery feature on the Master Indexer node. It is surely an awesome new feature that helps us solve a problem. We operate in a elastic cloud environment, and this will help us more easily deal with forwarder config of instances coming/going.

So my question is what are the pros and cons to using it? My intent is to go into building this with the feedback of others that have been through it and have learned!

[khourihan_splun]

Pros:

• Nice thing is that you don't have to update your forwarder app every time you add / remove and indexer - Peers add themselves into the pool
• Weighted Load balancing to spray the data on the indexers with most disk, so no fiddling with putting indexers in detention (most of the time

Cons:

• You have to setup and manage a pass4symmkey
• You have to allow the forwarders to talk to the clustermaster, which could introduce load on a busy CM

See this doc link for all the details: http://docs.splunk.com/Documentation/Splunk/7.0.0/Indexer/useforwarders#Advantages_of_the_indexer_di...

[ddrillic]

-- You have to allow the forwarders to talk to the clustermaster, which could introduce load on a busy CM

We are introducing here a single point of failure... right?

[teagan-pacheco]

Yes and no. As described in https://docs.splunk.com/Documentation/Splunk/9.0.3/Indexer/indexerdiscovery

 If the manager node goes down, the forwarders will rely on their most recent list of available peer nodes. However, the list does not persist through a forwarder restart. Therefore, if a forwarder restarts while the manager is down, it will not have a list of peer nodes and will not be able to forward data, resulting in potential data loss. Similarly, if a forwarder starts up for the first time, it must wait for the manager to return before it can get a list of peers.

Also, when using indexer discovery, each peer node can have only a single configured receiving port. The port can be configured for either [splunktcp] or [splunktcp-ssl], but not for both. You must use the same method for all peer nodes in the cluster: [splunktcp] or [splunktcp-ssl].