How do I configure the outputs.conf file to forward data from heavy forwarders to a group of load balanced indexers?

[thomas_forbes]

I have a cluster with a search head, master node, 2 indexers, and a deployment server. I am able to get the cluster to see new clients and push down updated .conf files, but I am having trouble having the forwarders actually send data to the peer nodes for indexing. The Admin documentation is not helping.

[muebel · Accepted solution]

Hi thomas.forbes, You can add each of the indexers to a tcpout stanza in outputs.conf on the forwarders, and make that the default tcpout like so:

[tcpout]
defaultGroup = indexers

[tcpout:indexers]
server = indexer1:9997, indexer2:9997

You can streamline this by using a A record containing all the indexers. Splunk will do DNS resolution to figure out all the entires. In this way, if you add indexers at a later point you won't have to update the splunk config, just the A record. i.e.:

[tcpout]
defaultGroup = indexers

[tcpout:indexers]
server = indexers.yoursite.com:9997

More info on outputs.conf can be found here : http://docs.splunk.com/Documentation/Splunk/latest/Admin/Outputsconf

Please let me know if this was helpful, and if you have any more questions 😄

[thomas_forbes]

Muebel,

That is exactly how I set-up my outputs.conf initially, with the following exception: "autoLB = true". For some reason this setting caused indexing to completely stop. All seems good now.

Thanks for the input,
Tom Forbes

[Jarohnimo]

How would you set this up in an index cluster? What would the outputs.conf file look like?

[esix_splunk]

One thing to note about DNS, you need to be aware and conscious the DNS TTL for the A records you create. This value should be set low, otherwise you will have sticky DNS and your may not balance across the indexers methodically.

That being said, including all of your indexers in a autoLB group in your outputs.conf is in lines of best practices.

[somesoni2]

See this as well.