Getting Data In

Forward Data to a third-party system

shangshin
Builder

Hi,
I am using SplunkForwarder to forward data to a non-splunk system by adding the stanza below in outputs.conf.

To verify this, I log onto the server (e.g. 10.1.1.35) and run a unix command lsof | grep 6996

However, I don't see anything returned. Is there anything I miss? Thanks in advance!

[tcpout:fastlane]
server = 10.1.1.35:6996
sendCookedData = false
1 Solution

alacercogitatus
SplunkTrust
SplunkTrust

Ok, so you will need to add this information into props and transforms. A quick reboot, and you should be set.

props.conf
[fastlane]
TRANSFORMS-forward = fastlaneTcpOut

transforms.conf
[fastlaneTcpOut]
REGEX = .
DEST_KEY=_TCP_ROUTING
FORMAT=fastlane

View solution in original post

acfecondo75
Path Finder

On a universal forwarder, in outputs.conf, you can change the defaultGroup to determine where ALL data is routed to by default.

outputs.conf:

[tcpout:third_party]
server=<ip>:<port>
sendCookedData=false

[tcpout]
defaultGroup=third_party

You can also define multiple server groups and put a comma separated list of them for your defaultGroup if you want to send data to multiple. This way you could send the data to Splunk and the third party.

[tcpout:third_party]
server=<ip>:<port>
sendCookedData=false

[tcpout:splunk_idx]
server=<ip>:<port>

[tcpout]
defaultGroup=third_party, splunk_idx

For reference, defaultGroup is the default setting for the _TCP_ROUTING key that you manipulate via props and transforms on the HF to do this kind of routing on a source, host, or sourcetype basis.

If you send the data to a HF or convert the box to a full splunk install, you can route via props and transforms. Similar to how you can specify multiple dests with the previous example, you can do that on the HF if you want too by specifying a comma separated list of dests in the FORMAT attribute.

0 Karma

vnguyen46
Contributor

in outputs.conf, don't forget to add [tcpout] on top if it is not already there - like when you create a fresh new file.

Thanks,

0 Karma

alacercogitatus
SplunkTrust
SplunkTrust

Ok, so you will need to add this information into props and transforms. A quick reboot, and you should be set.

props.conf
[fastlane]
TRANSFORMS-forward = fastlaneTcpOut

transforms.conf
[fastlaneTcpOut]
REGEX = .
DEST_KEY=_TCP_ROUTING
FORMAT=fastlane

alacercogitatus
SplunkTrust
SplunkTrust

I would place them in /splunk/etc/system/local. If you see other sourcetypes you may want to find /splunk/etc -name "outputs.conf and make sure nothing else is configured.

0 Karma

shangshin
Builder

BTW, If I place all 3 config files at /splunk/etc/apps/SplunkForwarder/default, there will be no data sending out from the "heavy forwarder"

0 Karma

shangshin
Builder

updated props.conf as
[my_apache_access]
TRANSFORMS-routing = apachelog

Unfortunately, I still see all sourcetypes sending over TCP.
Should I place the updated files in local or default?
splunk/etc/system/local or splunk/etc/system/default

It shouldn't matter but jsut can't find out what's the right solution.

0 Karma

alacercogitatus
SplunkTrust
SplunkTrust

In your props.conf the stanza is the sourcetype so the way you have it won't work, just remove the sourcetype:: and you should be all set.

0 Karma

shangshin
Builder

YES, it's working on the "Heavy Forwarder".
One last question -- How can I limit the data to certain sourcetype only e.g. my_apache_access? Will this work? Thanks again!

In props.conf
[sourcetype::my_apache_access]
TRANSFORMS-nyc = apachelog

In transforms.conf
[apachelog]
REGEX = .
DEST_KEY=_TCP_ROUTING
FORMAT=apachelogreader

In outputs.conf
[tcpout:apachelogreader]
server=10.1.1.197:7999
sendCookedData=false

0 Karma

saifuddin9122
Path Finder

hello

i am in the same situation you faced. i am trying to forward the data from splunk (indexer) to non-splunk system.

at present as a non-splunk system am using a linux machine.
can you please help me how exactly i should achieve the goal

0 Karma

alacercogitatus
SplunkTrust
SplunkTrust

Nope, a "Heavy Forwarder" was basically an indexer that sent cooked data to the main indexers. You couldn't search a heavy forwarder, all the data was on the indexer. You will want to place the outputs.conf, props.conf, transforms.conf stanzas onto your Splunk Indexer(s).

0 Karma

shangshin
Builder

My splunk version is Splunk 4.3.1 (build 119532)

On the remote server, I am using Splunk Universal Forwarder 4.3.4 (build 136012)

So to do the third party output, do I need to install "Heavy Forwarder" on the splunk server?

0 Karma

alacercogitatus
SplunkTrust
SplunkTrust

Yes. There used to be such a thing as a "Heavy Forwarder", which will in fact do the third party output. However, the UniversalForwarder (which I assumed you were using, I should have asked) will not support that. Relevant part: you can configure a heavy forwarder to route data conditionally to third-party systems.

What is your current version of Splunk? Are you using a Universal Forwarder?

0 Karma

shangshin
Builder

Do you mean I have to run this on the splunk server, the one that receives data from the splunkforwarder?

I am confused because the document says
Splunk forwarders can forward raw data to non-Splunk systems.
http://docs.splunk.com/Documentation/Splunk/5.0/Deploy/Forwarddatatothird-partysystemsd

0 Karma

alacercogitatus
SplunkTrust
SplunkTrust

You can't do this on the splunkforwarder, it has to be done on the indexer because of the use of TRANSFORMS (which is done at index time). Sorry, I missed that earlier.

0 Karma

shangshin
Builder

I don't use any special software to receive the data on linux but I did try to use linux command nc -l -p 6996 to listen on TCP port 6996

Is there a way to see log from the splunkforwarder for troubleshooting?

0 Karma

alacercogitatus
SplunkTrust
SplunkTrust

Well, what software is receiving this data on the non-splunk system server? Is it listening?

0 Karma

shangshin
Builder

I just added the information accordingly and restarted the splunkforwarder. I tried to verify on the server of non-splunk system by using lsof | grep 6996 Howeever, nothing returns.

Is there a way to verfiy the raw data is successfully forwarded to the non-splunk system server? Thanks again!

0 Karma

shangshin
Builder

It's empty in props and transforms since I want to send all the data from this splunkforwarder.

However, I do have the stanza in inputs.conf

[monitor:///local/0/access.log]
host = myremosthost
sourcetype = fastlane
index = apache_access_log
disabled = 0

0 Karma

alacercogitatus
SplunkTrust
SplunkTrust

Please post the props and transforms for the sourcetype "fastlane".

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...