Getting Data In

Can you forward specific data to a third-party system?

jeffbat
Path Finder

I am working on a POC third-party system for some of our data and need to get data from Splunk forwarded over to it.

I was looking through this link http://docs.splunk.com/Documentation/Splunk/6.6.3/Forwarding/Forwarddatatothird-partysystemsd

And was hoping someone might have done what I am trying to do.

We want to send all of our Windows & IIS logs from our forwarders to the third-party system as a syslog feed.

All of our forwarders currently send directly to our backend indexers (which are a set of 3 different indexer clusters).

From looking at that link, it seems like if I want to separate data (only some sourcetypes/indexes/etc) that is getting sent from the forwarders to the other location, I have to pass the data through a heavy forwarder. I want to avoid doing this because that would mean repointing all of our forwarders to go through the heavy forwarder.

Can the division of the data be done from the forwarders themselves? Or even by making a change on the indexer side to get the raw data over to the third-party through a syslog feed?

0 Karma
1 Solution

horsefez
Motivator

Hi @jeffbat,

that doesn't necessarily mean that you have to let all the forwards go through a HF. Ofc, only the forwarders that actually collect Windows & IIS logs are required to do so.

The Heavy Forwarder (HF) has the ability to filter the traffic on exactly the logfiles you want to send to another system. If you are trying to avoid using one you only really can send the data from the Universal Forwarders (UF) directly to another (additional) system. (two output targets: one your splunk instance, one your other product)

The caveat with the UF is that you will send all the data that is collected on the system. You won't be able to filter on only Windows & IIS logs.
If you have a chance to filter it at the receiving end on the other solution, this might be the only way you can avoid using a HF.

View solution in original post

0 Karma

horsefez
Motivator

Hi @jeffbat,

that doesn't necessarily mean that you have to let all the forwards go through a HF. Ofc, only the forwarders that actually collect Windows & IIS logs are required to do so.

The Heavy Forwarder (HF) has the ability to filter the traffic on exactly the logfiles you want to send to another system. If you are trying to avoid using one you only really can send the data from the Universal Forwarders (UF) directly to another (additional) system. (two output targets: one your splunk instance, one your other product)

The caveat with the UF is that you will send all the data that is collected on the system. You won't be able to filter on only Windows & IIS logs.
If you have a chance to filter it at the receiving end on the other solution, this might be the only way you can avoid using a HF.

0 Karma

jeffbat
Path Finder

Pretty much all of our forwarders are are the ones that have Windows logs (almost all) & IIS logs (many).

Can the filtering piece be done on the indexer cluster peer node level instead of having to hit a HF first?

0 Karma

jeffbat
Path Finder

I was discussing this with our Splunk engineer and the bigger caveat that came out of it was since the data needed to be sent to the third-party in syslog format, the only way to do that was to utilize a HF.

It is disappointing that I have to have another server out there just so that I can have the data sent out to the third-party.

0 Karma
Get Updates on the Splunk Community!

How to Monitor Google Kubernetes Engine (GKE)

We’ve looked at how to integrate Kubernetes environments with Splunk Observability Cloud, but what about ...

Index This | How can you make 45 using only 4?

October 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this ...

Splunk Education Goes to Washington | Splunk GovSummit 2024

If you’re in the Washington, D.C. area, this is your opportunity to take your career and Splunk skills to the ...