topic Re: How to configure a heavy forwarder to reroute data from a universal forwarder into a special index different from the one specified in the UF's inputs.conf? in Getting Data In

How to configure a heavy forwarder to reroute data from a universal forwarder into a special index different from the one specified in the UF's inputs.conf?

tpaulsen — Tue, 01 Sep 2015 14:40:16 GMT

Hi,

We need some help with rerouting data from a universal forwarder via a heavy forwarder into a different index than given in the UF inputs.conf.

Universal Forwarder inputs.conf:

[monitor:///var/opt/log/tomcat/test.log]
disabled = false
sourcetype = testdatenb2c

Heavy Forwarder (as man in the middle) configuration:
inputs.conf:

[default]
host = splunk-heavy-dev-369368
[splunktcp://9697]
connection_host = ip

props.conf:

[testdatenb2c]
TRANSFORMS-routing = transforms_TEST

transforms.conf

[transforms_TEST]
DEST_KEY=_TCP_ROUTING
FORMAT=B2CDATEN2EC
REGEX=.

outputs.conf

[tcpout:B2CDATEN2EC]
#Disable the current filters from the defaults outputs.conf
forwardedindex.0.whitelist =
forwardedindex.1.blacklist =
forwardedindex.2.whitelist =

forwardedindex.0.whitelist = test
disabled = false
server = my-splunk-idx-dev:9997

Now, the data gets indexed by the my-splunk-idx-dev server, but the data doesn't land in the index "test". Instead, it is being indexed in "main" as the default configured on the UF.
What are we doing wrong here? How can we reroute the data to our "test" index on the my-splunk-idx-dev server?

Thank you in advance for any help,

Best regards, Tom

Re: How to configure a heavy forwarder to reroute data from a universal forwarder into a special index different from the one specified in the UF's inputs.conf?

somesoni2 — Tue, 01 Sep 2015 15:12:29 GMT

The outputs.conf forwardedindex whitelist setting only does the filter of events based on index name but doesn't update the index name in the data. That has to be done in the props-transforms conf file.

Try this for your props.conf and transforms.conf in your Heavy forwarder
Updated fixed typo, added missing underscore

props.conf
[testdatenb2c]
TRANSFORMS-routing = transforms_routing,transforms_index

transforms.conf

[transforms_routing]
DEST_KEY=_TCP_ROUTING
FORMAT=B2CDATEN2EC
REGEX=.

[transforms_index]
DEST_KEY = _MetaData:Index
FORMAT = test
REGEX=.

Re: How to configure a heavy forwarder to reroute data from a universal forwarder into a special index different from the one specified in the UF's inputs.conf?

tpaulsen — Wed, 02 Sep 2015 09:03:21 GMT

Thank you very much! It worked! For others...note that it´s _MetaData (with leading underscore).

Re: How to configure a heavy forwarder to reroute data from a universal forwarder into a special index different from the one specified in the UF's inputs.conf?

wkupersa — Fri, 06 Oct 2017 19:50:08 GMT

Thanks to both of you. Very helpful!