topic Re: How do I setup TCP ROUTING when using an intermediate forwarder in Getting Data In

How do I setup TCP ROUTING when using an intermediate forwarder

pkeller — Tue, 17 Mar 2015 17:19:22 GMT

I have some sourcetypes that I'd like to go to two indexing destinations.

Universal Forwarder: (inputs.conf)

[monitor:///path/to/logs]
index=myindex
sourcetype=mysourcetype
_TCP_ROUTING=oregon,sanDiego

Intermediate forwarder: (outputs.conf)

[tcpout]
defaultGroup=oregon

tcpout:oregon]
autoLB = true
autoLBFrequency = 30
server = portland1:7777,portland2:7777,portland3:7777

[tcpout:sanDiego]
autoLB = true
autoLBFrequency = 30
server = sd1:7777,sd2:7777,sd3:7777

The logs are going only to the default group (oregon), so I'm wondering now if I need to add forwarding instances on my intermediate forwarders ... so that one forwarding instance routes to oregon and the other to san diego ... thus I wouldn't have any tcp routing statements on the intermediate forwarder ...

I'm hoping that I'm making sense here ...

ie: (inputs.conf)

[monitor:///path/to/logs]
index=myindex
sourcetype=mysourcetype
_TCP_ROUTING=oregon_forwarders,sandiego_forwarders

then: (also on the UF) - outputs.conf

[tcpout]
defaultGroup=oregon_forwarders

tcpout:oregon_forwarders]
autoLB = true
autoLBFrequency = 30
server = forwarder1:7777,forwarder2:7777

[tcpout:sandiego_forwarders]
autoLB = true
autoLBFrequency = 30
server = forwarder1:7778,forwarder2:7778

Re: How do I setup TCP ROUTING when using an intermediate forwarder

theouhuios — Tue, 17 Mar 2015 17:26:57 GMT

You want to index them at oregon? Or do you want oregon to just act as forwarder and push those logs to sanDiego?

I do this in our environment where I route prod server logs to dev environment through forwarders in the middle which have ports and fw rules open to dev. Not sure if this is same type of configuration that you are trying to achieve.

Re: How do I setup TCP ROUTING when using an intermediate forwarder

pkeller — Tue, 17 Mar 2015 17:36:02 GMT

I want to index them at both locations ... mainly for testing purposes as we're experiencing a small amount of corruption, so I'm looking to see whether both sites experiencing the same corruption or whether it's something in the network.

Thanks very much.

Re: How do I setup TCP ROUTING when using an intermediate forwarder

theouhuios — Tue, 17 Mar 2015 17:45:49 GMT

http://docs.splunk.com/Documentation/Splunk/5.0.4/Deploy/Configureforwarderswithoutputs.confd#Types_of_outputs.conf_files

You should look at the Data Cloning part in the link.

Re: How do I setup TCP ROUTING when using an intermediate forwarder

pkeller — Tue, 17 Mar 2015 17:52:43 GMT

Thank you ... That's perfect.