How to send different logs to different indexers from the same Universal Forwarder?

[hartfoml]

I have one universal forwarder (UF) that is sending production data to the production intermediate Forwarder (IF) and then on to the production indexers.

I would like to start collecting test data from the UF and send it to the LAB indexer.

Is there any way to change the outputs.conf for one app on the UF and leave the outputs.conf that is presently sending live data to the IF?

[ragedsparrow · Accepted solution]

You'll want to specify TCP_ROUTING in your inputs. So, I'm assuming that you will have separate outputs.conf files with separate tcpout stanzas. What you'll want to do in your inputs.conf file is specify which tcpout stanza to use:

outputs.conf for app1:

[tcpout:IndexerA]
 server=192.168.56.101:8089
 ....
 ....

outputs.conf for app2:

[tcpout:IndexerB]
 server=192.168.56.102:8089
 ....
 ....

In your inputs.conf for each of the apps, you'll specify which tcpout stanza to use with TCP_ROUTING:

inputs.conf for app1

[monitor:///path/to/log/A/logA.log]
 # Add attributes to your monitor like sourcetype, index, etc
 ....
 ....
 # In the end, specify to which indexer this log should be sent using _TCP_ROUTING = <group name>
 _TCP_ROUTING = IndexerA

inputs.conf for app2:

[monitor:///path/to/log/B/logB.log]
 ....
 ....
 _TCP_ROUTING = IndexerB

I don't send data to an intermediate forwarder, but this is how I send separate data to separate indexers. Let me know if this helps.

References:
http://docs.splunk.com/Documentation/Splunk/6.3.0/admin/inputsconf
http://docs.splunk.com/Documentation/Splunk/6.3.0/admin/Outputsconf
http://docs.splunk.com/Documentation/Splunk/6.3.0/Forwarding/Configureforwarderswithoutputs.confd

[jaxjohnny2000]

Hi. would you mind looking at this configuration also please? I'm missing something in the flow, or should this be working?

inputs.conf:
[monitor:///syslog/logs/proxy_other//.log]
host_segment = 4

sourcetype = mcafee:wg:kv

sourcetype = MWGaccess3
index = proxy_index
_TCP_ROUTING = main_indexcluster_idx #main index cluster need a copy here for Splunk
_SYSLOG_ROUTING = mskysap_syslog_group #alternate source a copy here for another syslog server

Props.conf:
[source::///syslog/logs/proxy_other//.log]
TRANSFORMS-mskysap = send_syslog_to_mskysap

transforms.conf:
[send_syslog_to_mskysap]
REGEX = .
DEST_KEY = _SYSLOG_ROUTING
FORMAT = mskysap_syslog_group

outputs.conf
[syslog:mskysap_syslog_group]
type = TCP
server = different_syslog_server:514

[hartfoml]

Thanks @ragedsparrow I haven't tried this but this is the correct answer. Look at you helping out after just signing up a little over a year ago. "Lowe's" is lucky to have you. Thanks very much for helping out. Keep up the good work.

[ragedsparrow]

I appreciate it @hartfoml . I have had to look this up in the past and had it in a reference document to be used again if ever I needed it. when I came across your question it was very similar to what I was running into a while ago, so I figured I'd try to help.