Forward Data to a third-party system

[shangshin]

Hi,
I am using SplunkForwarder to forward data to a non-splunk system by adding the stanza below in outputs.conf.

To verify this, I log onto the server (e.g. 10.1.1.35) and run a unix command lsof | grep 6996

However, I don't see anything returned. Is there anything I miss? Thanks in advance!

[tcpout:fastlane]
server = 10.1.1.35:6996
sendCookedData = false

[acfecondo75]

On a universal forwarder, in outputs.conf, you can change the defaultGroup to determine where ALL data is routed to by default.

outputs.conf:

[tcpout:third_party]
server=<ip>:<port>
sendCookedData=false

[tcpout]
defaultGroup=third_party

You can also define multiple server groups and put a comma separated list of them for your defaultGroup if you want to send data to multiple. This way you could send the data to Splunk and the third party.

[tcpout:third_party]
server=<ip>:<port>
sendCookedData=false

[tcpout:splunk_idx]
server=<ip>:<port>

[tcpout]
defaultGroup=third_party, splunk_idx

For reference, defaultGroup is the default setting for the _TCP_ROUTING key that you manipulate via props and transforms on the HF to do this kind of routing on a source, host, or sourcetype basis.

If you send the data to a HF or convert the box to a full splunk install, you can route via props and transforms. Similar to how you can specify multiple dests with the previous example, you can do that on the HF if you want too by specifying a comma separated list of dests in the FORMAT attribute.

[vnguyen46]

in outputs.conf, don't forget to add [tcpout] on top if it is not already there - like when you create a fresh new file.

Thanks,

[alacercogitatus · Accepted solution]

Ok, so you will need to add this information into props and transforms. A quick reboot, and you should be set.

props.conf
[fastlane]
TRANSFORMS-forward = fastlaneTcpOut


transforms.conf
[fastlaneTcpOut]
REGEX = .
DEST_KEY=_TCP_ROUTING
FORMAT=fastlane


[alacercogitatus]

I would place them in /splunk/etc/system/local. If you see other sourcetypes you may want to find /splunk/etc -name "outputs.conf and make sure nothing else is configured.

[shangshin]

BTW, If I place all 3 config files at /splunk/etc/apps/SplunkForwarder/default, there will be no data sending out from the "heavy forwarder"

[shangshin]

updated props.conf as
[my_apache_access]
TRANSFORMS-routing = apachelog

Unfortunately, I still see all sourcetypes sending over TCP.
Should I place the updated files in local or default?
splunk/etc/system/local or splunk/etc/system/default

It shouldn't matter but jsut can't find out what's the right solution.

[alacercogitatus]

In your props.conf the stanza is the sourcetype so the way you have it won't work, just remove the sourcetype:: and you should be all set.

[shangshin]

YES, it's working on the "Heavy Forwarder".
One last question -- How can I limit the data to certain sourcetype only e.g. my_apache_access? Will this work? Thanks again!

In props.conf
[sourcetype::my_apache_access]
TRANSFORMS-nyc = apachelog

In transforms.conf
[apachelog]
REGEX = .
DEST_KEY=_TCP_ROUTING
FORMAT=apachelogreader

In outputs.conf
[tcpout:apachelogreader]
server=10.1.1.197:7999
sendCookedData=false

[saifuddin9122]

hello

i am in the same situation you faced. i am trying to forward the data from splunk (indexer) to non-splunk system.

at present as a non-splunk system am using a linux machine.
can you please help me how exactly i should achieve the goal

[alacercogitatus]

Nope, a "Heavy Forwarder" was basically an indexer that sent cooked data to the main indexers. You couldn't search a heavy forwarder, all the data was on the indexer. You will want to place the outputs.conf, props.conf, transforms.conf stanzas onto your Splunk Indexer(s).

[shangshin]

My splunk version is Splunk 4.3.1 (build 119532)

On the remote server, I am using Splunk Universal Forwarder 4.3.4 (build 136012)

So to do the third party output, do I need to install "Heavy Forwarder" on the splunk server?

[alacercogitatus]

Yes. There used to be such a thing as a "Heavy Forwarder", which will in fact do the third party output. However, the UniversalForwarder (which I assumed you were using, I should have asked) will not support that. Relevant part: you can configure a heavy forwarder to route data conditionally to third-party systems.

What is your current version of Splunk? Are you using a Universal Forwarder?

[shangshin]

Do you mean I have to run this on the splunk server, the one that receives data from the splunkforwarder?

I am confused because the document says
Splunk forwarders can forward raw data to non-Splunk systems.
http://docs.splunk.com/Documentation/Splunk/5.0/Deploy/Forwarddatatothird-partysystemsd

[alacercogitatus]

You can't do this on the splunkforwarder, it has to be done on the indexer because of the use of TRANSFORMS (which is done at index time). Sorry, I missed that earlier.

[shangshin]

I don't use any special software to receive the data on linux but I did try to use linux command nc -l -p 6996 to listen on TCP port 6996

Is there a way to see log from the splunkforwarder for troubleshooting?

[alacercogitatus]

Well, what software is receiving this data on the non-splunk system server? Is it listening?

[shangshin]

I just added the information accordingly and restarted the splunkforwarder. I tried to verify on the server of non-splunk system by using lsof | grep 6996 Howeever, nothing returns.

Is there a way to verfiy the raw data is successfully forwarded to the non-splunk system server? Thanks again!

[shangshin]

It's empty in props and transforms since I want to send all the data from this splunkforwarder.

However, I do have the stanza in inputs.conf

[monitor:///local/0/access.log]
host = myremosthost
sourcetype = fastlane
index = apache_access_log
disabled = 0

[alacercogitatus]

Please post the props and transforms for the sourcetype "fastlane".

[NeharikaVats]

Knowing that its a very very old thread, still at this hour we are stuck with the same issue. So we are trying to send the data from Splunk heavy forwarder to a third party destination and the connectivity looks fine but the outputs, props & transforms config that we have done does not seem to be working. I need an urgent help at this case if somebody can ???

This is how our config looks like now:

outputs.conf:

[tcpout]

defaultGroup = nothing

disabled = false

[tcpout:datab]

server = x.x.x.x:xxxx

sendCookedData = false

compressed = false

props.conf

[host::x.x.x.x]

TRANSFORMS-x.x.x.x = route_datab

#SHOULD_LINEMERGE = false

transforms.conf

[route_datab]

REGEX = .

DEST_KEY = _TCP_ROUTING

FORMAT = datab