topic Re: Forwarding select data in my environment. in Getting Data In

Forwarding select data in my environment.

Steve_Litras — Fri, 13 Aug 2010 04:56:33 GMT

I'm trying to take data from specific systems and, after indexing it, forward it to a third party for other analysis. I have data coming from light forwarders on the systems in question, and going through a forwarding layer before it gets to the indexing hosts. I applied, basically, the recipe as defined in the documentation on the indexers:

props.conf:

[host::*WCA*]
TRANSFORMS-routing = send_to_syslog

transforms.conf (note, splunk complains about not having a REGEX entry - I've tried it with .* as the REGEX to no avail):

[send_to_syslog]
DEST_KEY = _SYSLOG_ROUTING
FORMAT = SW_syslog_group

outputs.conf:

[syslog:SW_syslog_group]
disabled=false
server = 10.20.30.40:514

I'm wondering if I need to apply the props.conf and the transforms.conf at the forwarder layer, and if I do that, will it still properly index, or will it forward out prior to indexing?

thanks Steve

Re: Forwarding select data in my environment.

gkanapathy — Fri, 13 Aug 2010 06:30:13 GMT

So you have LWF->Intermediate Hvy Forwarder->Indexer? If so, the problem is that the TRANSFORMS/parsing phase will not occur on the indexer. However, you're right that you don't want to set this on the intermediate forwarder layer either, because I'm pretty sure it won't carry through to the indexer. I would try it though, because if it does work it will be a lot easier.

Otherwise, I'd would instead set up an extra Splunk TCP port on the indexer:

# regular tcp port
[splunktcp://9997]

# special tcp port everything sent here goes to syslog out
[splunktcp://9999]
_SYSLOG_ROUTING = SW_syslog_group

Then on the intermediate layer, you route to the appropriate Splunk TCP output group:

[send_to_syslog_port]
DEST_KEY = _TCP_ROUTING
FORMAT = SW_splunk_syslog_port_group

where the SW_splunk_syslog_port_group goes to port 9999. (The default group is the one that goes to 9997.)

It's also possible to force the indexer to run parsed data (from the Hvy Forwarder) through the parse queue again. This has problems, depending on your sourcetype rules, because some transforms might get run more than once, which means you'll need to go through configs on the indexers and disable default or previously inactive configs to prevent that. Once you've done that, you can add into a global inputs.conf:

[splunktcp]
route=has_key:_utf8:parsingQueue;has_key:_linebreaker:parsingQueue;absent_key:_utf8:parsingQueue;absent_key:_linebreaker:parsingQueue

This overrides the default rule that only parses data from a LWF, and parses everything from any forwarder. I'm not sure if you could do:

[splunktcp://9999]
route=has_key:_utf8:parsingQueue;has_key:_linebreaker:parsingQueue;absent_key:_utf8:parsingQueue;absent_key:_linebreaker:parsingQueue

and only apply it to data on one port. If it does work, and if you need or want to distinguish it, that might not be a bad idea. This isn't great, because you're put

Re: Forwarding select data in my environment.

Steve_Litras — Fri, 13 Aug 2010 22:46:16 GMT

If I set up the alternate port, as you suggested, would the data still be indexed? Do I, in that case need to specify:

indexAndForward = true

for the "special" splunktcp port?

Re: Forwarding select data in my environment.

Steve_Litras — Tue, 21 Sep 2010 05:41:59 GMT

Ok - so I finally was able to get this in place (change windows and firewalls oh my), and it works, but the data being sent to the 3rd party no longer has the original host header in it. Is that due to truncation, or is there something I can do to specifically make sure that data goes with the data?