Deployment Architecture

How do I configure the outputs.conf file to forward data from heavy forwarders to a group of load balanced indexers?

Communicator

I have a cluster with a search head, master node, 2 indexers, and a deployment server. I am able to get the cluster to see new clients and push down updated .conf files, but I am having trouble having the forwarders actually send data to the peer nodes for indexing. The Admin documentation is not helping.

1 Solution

SplunkTrust
SplunkTrust

Hi thomas.forbes, You can add each of the indexers to a tcpout stanza in outputs.conf on the forwarders, and make that the default tcpout like so:

[tcpout]
defaultGroup = indexers

[tcpout:indexers]
server = indexer1:9997, indexer2:9997

You can streamline this by using a A record containing all the indexers. Splunk will do DNS resolution to figure out all the entires. In this way, if you add indexers at a later point you won't have to update the splunk config, just the A record. i.e.:

[tcpout]
defaultGroup = indexers

[tcpout:indexers]
server = indexers.yoursite.com:9997

More info on outputs.conf can be found here : http://docs.splunk.com/Documentation/Splunk/latest/Admin/Outputsconf

Please let me know if this was helpful, and if you have any more questions 😄

View solution in original post

SplunkTrust
SplunkTrust

Hi thomas.forbes, You can add each of the indexers to a tcpout stanza in outputs.conf on the forwarders, and make that the default tcpout like so:

[tcpout]
defaultGroup = indexers

[tcpout:indexers]
server = indexer1:9997, indexer2:9997

You can streamline this by using a A record containing all the indexers. Splunk will do DNS resolution to figure out all the entires. In this way, if you add indexers at a later point you won't have to update the splunk config, just the A record. i.e.:

[tcpout]
defaultGroup = indexers

[tcpout:indexers]
server = indexers.yoursite.com:9997

More info on outputs.conf can be found here : http://docs.splunk.com/Documentation/Splunk/latest/Admin/Outputsconf

Please let me know if this was helpful, and if you have any more questions 😄

View solution in original post

Communicator

Muebel,

That is exactly how I set-up my outputs.conf initially, with the following exception: "autoLB = true". For some reason this setting caused indexing to completely stop. All seems good now.

Thanks for the input,
Tom Forbes

0 Karma

Builder

How would you set this up in an index cluster? What would the outputs.conf file look like?

0 Karma

Splunk Employee
Splunk Employee

One thing to note about DNS, you need to be aware and conscious the DNS TTL for the A records you create. This value should be set low, otherwise you will have sticky DNS and your may not balance across the indexers methodically.

That being said, including all of your indexers in a autoLB group in your outputs.conf is in lines of best practices.

Revered Legend

See this as well.

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!