Can you forward specific data to a third-party system?

[jeffbat]

I am working on a POC third-party system for some of our data and need to get data from Splunk forwarded over to it.

I was looking through this link http://docs.splunk.com/Documentation/Splunk/6.6.3/Forwarding/Forwarddatatothird-partysystemsd

And was hoping someone might have done what I am trying to do.

We want to send all of our Windows & IIS logs from our forwarders to the third-party system as a syslog feed.

All of our forwarders currently send directly to our backend indexers (which are a set of 3 different indexer clusters).

From looking at that link, it seems like if I want to separate data (only some sourcetypes/indexes/etc) that is getting sent from the forwarders to the other location, I have to pass the data through a heavy forwarder. I want to avoid doing this because that would mean repointing all of our forwarders to go through the heavy forwarder.

Can the division of the data be done from the forwarders themselves? Or even by making a change on the indexer side to get the raw data over to the third-party through a syslog feed?

[horsefez · Accepted solution]

Hi @jeffbat,

that doesn't necessarily mean that you have to let all the forwards go through a HF. Ofc, only the forwarders that actually collect Windows & IIS logs are required to do so.

The Heavy Forwarder (HF) has the ability to filter the traffic on exactly the logfiles you want to send to another system. If you are trying to avoid using one you only really can send the data from the Universal Forwarders (UF) directly to another (additional) system. (two output targets: one your splunk instance, one your other product)

The caveat with the UF is that you will send all the data that is collected on the system. You won't be able to filter on only Windows & IIS logs.
If you have a chance to filter it at the receiving end on the other solution, this might be the only way you can avoid using a HF.

[jeffbat]

Pretty much all of our forwarders are are the ones that have Windows logs (almost all) & IIS logs (many).

Can the filtering piece be done on the indexer cluster peer node level instead of having to hit a HF first?

[jeffbat]

I was discussing this with our Splunk engineer and the bigger caveat that came out of it was since the data needed to be sent to the third-party in syslog format, the only way to do that was to utilize a HF.

It is disappointing that I have to have another server out there just so that I can have the data sent out to the third-party.