Can you configure the Universal Forwarder on NIX (syslog) to send some logs to the indexer and others to a Heavy Forwarder?

[HackerHurricane]

We have a syslog server where there are many logs going to the indexer.

Can we configure the Linux Universal Forwarder to send some files to the indexer and others to the Heavy Forwarder to be post processed, thinned out is the goal?

How best to do this?

Two Univ Forwarder instances?

Or can a config deal with this?

Thanks,

MG

[jtacy]

What you want to do is definitely possible but I'm curious what other options you've looked at since this will add a little bit of complexity to your environment. I would probably consider doing some filtering/routing on the syslog server itself but if that wasn't an option I'd try filtering on the indexer. I'm concerned that you might not get the expected benefits unless you're going to be able to filter a large percentage of events at the heavy forwarder. It may depend on the event size, number of events per second, and also what specific problem you're trying to work around by filtering the events.

If you do want to go ahead and split the traffic, on a single UF you can create tcpout stanzas for your indexer and heavy forwarder, then when you need to route to your heavy forwarder you can use the _TCP_ROUTING option. It would look something like this (tested on UF 6.4.0 on Linux x64):

outputs.conf

[tcpout]
defaultGroup = indexer

[tcpout:indexer]
server = indexerhost:port

[tcpout:heavyforwarder]
server = heavyforwarderhost:port

inputs.conf

[monitor:///opt/splunk/app/good.txt]
index = main
sourcetype = good

[monitor:///opt/splunk/app/noisy.txt]
index = main
sourcetype = noisy
_TCP_ROUTING = heavyforwarder

# Route internal events to the indexer only to avoid duplicates.
# UF sends to all configured tcpout groups on these inputs by default.
[monitor://$SPLUNK_HOME/var/log/splunk/splunkd.log]
_TCP_ROUTING = indexer

[monitor://$SPLUNK_HOME/var/log/splunk/metrics.log]
_TCP_ROUTING = indexer

[monitor://$SPLUNK_HOME/etc/splunk.version]
_TCP_ROUTING = indexer

The last set of inputs isn't required but you'll end up with some duplicate events in _internal if you don't make the change; you'll be overriding some default UF behavior where _TCP_ROUTING = * on certain inputs so keep in mind that the net effect of this could vary by UF version.

Note that because defaultGroup = indexer is set in outputs.conf, you only need to set _TCP_ROUTING = heavyforwarder when you need to override the default and won't have to change the rest of your inputs.

Relevant docs: http://docs.splunk.com/Documentation/Splunk/6.4.3/Forwarding/Routeandfilterdatad