topic Re: What is best practices for forwarders and an index cluster? in Getting Data In

What is best practices for forwarders and an index cluster?

borja_luaces — Mon, 18 Nov 2019 12:58:06 GMT

Good morning all,

I am building a lab environment at AWS and I would like to know which one is the best approach for sending the logs from the forwarders.

Based on the image which one will be the right approach?

A - Send the logs from the forwarders into the master cluster and the master cluster forwards them to the indexers?
B - Send the logs to the 1st indexer and the master cluster will handle the replication?
C - Send the logs to any indexer and the master cluster will handle the replication?
D - None of the above

I have tried to look for documentation regarding this issue but have not been lucky finding it, if anyone can point me in the right way I will appreciate it.

Thanks a lot all for your time

Re: What is best practices for forwarders and an index cluster?

richgalloway — Mon, 18 Nov 2019 14:31:59 GMT

All forwarders should send their data to indexers, not to the Cluster Master. The indexers will perform replication.
Forwarders should take advantage of the Indexer DIscovery feature there they ask the Cluster Master for a list of indexers to which to send data.
Documentation for the UF is at https://docs.splunk.com/Documentation/Forwarder/8.0.0/Forwarder/Abouttheuniversalforwarder.
See https://docs.splunk.com/Documentation/Forwarder/8.0.0/Forwarder/Forwarderdeploymenttopologies for how forwarders connect to indexers.
See https://docs.splunk.com/Documentation/Forwarder/8.0.0/Forwarder/Configureforwardingtoindexerclusters#Connect_forwarders_to_peer_nodes for details about Indexer Discovery

Re: What is best practices for forwarders and an index cluster?

borja_luaces — Mon, 18 Nov 2019 14:37:11 GMT

Thank for the fast reply.

I will that documentation ASAP.

I might have more questions about forwarders and indexers so I will be updating this post

Re: What is best practices for forwarders and an index cluster?

gcusello — Mon, 18 Nov 2019 14:37:49 GMT

Hi @borja_luaces,
as you said there are two approaches:

you can address each Indexer in outputs.conf file of each Universal Forwarder (the best approacch is to manage outputs.conf in a dedicated App to deploy with Deployment Server), but in this way, if you add a new Indexer you have to modify outputs.conf and deploy it;
you can address the Master Node (not sending logs to it!) and it says to Universal Forwarders the active Indexers, in this way you don't need to modify outputs.conf in every Universal Forwarder.

I hint the second solution!
You can find more infos at:
https://docs.splunk.com/Documentation/Splunk/8.0.0/Indexer/useforwarders
https://docs.splunk.com/Documentation/Splunk/8.0.0/Indexer/indexerdiscovery

Ciao.
Giuseppe