Deployment Architecture

How do I configure the outputs.conf file to forward data from heavy forwarders to a group of load balanced indexers?

thomas_forbes
Communicator

I have a cluster with a search head, master node, 2 indexers, and a deployment server. I am able to get the cluster to see new clients and push down updated .conf files, but I am having trouble having the forwarders actually send data to the peer nodes for indexing. The Admin documentation is not helping.

1 Solution

muebel
SplunkTrust
SplunkTrust

Hi thomas.forbes, You can add each of the indexers to a tcpout stanza in outputs.conf on the forwarders, and make that the default tcpout like so:

[tcpout]
defaultGroup = indexers

[tcpout:indexers]
server = indexer1:9997, indexer2:9997

You can streamline this by using a A record containing all the indexers. Splunk will do DNS resolution to figure out all the entires. In this way, if you add indexers at a later point you won't have to update the splunk config, just the A record. i.e.:

[tcpout]
defaultGroup = indexers

[tcpout:indexers]
server = indexers.yoursite.com:9997

More info on outputs.conf can be found here : http://docs.splunk.com/Documentation/Splunk/latest/Admin/Outputsconf

Please let me know if this was helpful, and if you have any more questions 😄

View solution in original post

muebel
SplunkTrust
SplunkTrust

Hi thomas.forbes, You can add each of the indexers to a tcpout stanza in outputs.conf on the forwarders, and make that the default tcpout like so:

[tcpout]
defaultGroup = indexers

[tcpout:indexers]
server = indexer1:9997, indexer2:9997

You can streamline this by using a A record containing all the indexers. Splunk will do DNS resolution to figure out all the entires. In this way, if you add indexers at a later point you won't have to update the splunk config, just the A record. i.e.:

[tcpout]
defaultGroup = indexers

[tcpout:indexers]
server = indexers.yoursite.com:9997

More info on outputs.conf can be found here : http://docs.splunk.com/Documentation/Splunk/latest/Admin/Outputsconf

Please let me know if this was helpful, and if you have any more questions 😄

thomas_forbes
Communicator

Muebel,

That is exactly how I set-up my outputs.conf initially, with the following exception: "autoLB = true". For some reason this setting caused indexing to completely stop. All seems good now.

Thanks for the input,
Tom Forbes

0 Karma

Jarohnimo
Builder

How would you set this up in an index cluster? What would the outputs.conf file look like?

0 Karma

esix_splunk
Splunk Employee
Splunk Employee

One thing to note about DNS, you need to be aware and conscious the DNS TTL for the A records you create. This value should be set low, otherwise you will have sticky DNS and your may not balance across the indexers methodically.

That being said, including all of your indexers in a autoLB group in your outputs.conf is in lines of best practices.

somesoni2
SplunkTrust
SplunkTrust

See this as well.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...