Getting Data In

How to edit my configurations to use Heavy Forwarder to filter and route data to another Splunk setup?

meidal_splunkin
New Member

Hi,

I'm trying to use Heavy Forwarders (HF) to route and filter data to another Splunk setup outside of mine. My goal is to send only sourcetype=log4net matching a REGEX (let's say ClientName). I managed to do this but the client requested that I change also the index to where I sent which totally messed up my solution.

Trying to make it short: index=main sourcetype=log4net with ClientName should be routed to the client, index=main sourcetype=iis whatever should not. Any help is deeply appreciated!

###props.conf
[default]
TRANSFORMS-def=clearlog
[log4net]
TRANSFORMS-routing=clearlog,client

###transforms.conf

[clearlog]
REGEX=.
DEST_KEY=queue
FORMAT=nullQueue

[germany]
REGEX=ClientName
DEST_KEY=_MetaData:Index
FORMAT=Clientwtv
0 Karma
1 Solution

somesoni2
SplunkTrust
SplunkTrust

This will be done in two steps.

1) Configure your HF to do the filtering for the sourcetype log4net to send specific events to Third Party Splunk instance (http://docs.splunk.com/Documentation/Splunk/6.5.2/Forwarding/Routeandfilterdatad#Replicate_a_subset_...). I'm guessing this is already done.

2) Change the destination index for this data on the Third party Splunk instance. Now, if you try to override it in your HF, it'll change index name for all events from log4net including it for your events being sent to your Splunk for that sourcetype. So this should be done at the third party Splunk receiver/indexer. Since HF has already parsed it, the third party Splunk instance will not parse it again. To make it do so, use the method defined here. https://answers.splunk.com/answers/97918/reparsing-cooked-data-coming-from-a-heavy-forwarder-possibl... . Then you setup your transforms for overriding index name on third party Splunk instance. https://answers.splunk.com/answers/246672/how-can-i-override-an-index-name-based-on-sourcety.html

View solution in original post

0 Karma

somesoni2
SplunkTrust
SplunkTrust

This will be done in two steps.

1) Configure your HF to do the filtering for the sourcetype log4net to send specific events to Third Party Splunk instance (http://docs.splunk.com/Documentation/Splunk/6.5.2/Forwarding/Routeandfilterdatad#Replicate_a_subset_...). I'm guessing this is already done.

2) Change the destination index for this data on the Third party Splunk instance. Now, if you try to override it in your HF, it'll change index name for all events from log4net including it for your events being sent to your Splunk for that sourcetype. So this should be done at the third party Splunk receiver/indexer. Since HF has already parsed it, the third party Splunk instance will not parse it again. To make it do so, use the method defined here. https://answers.splunk.com/answers/97918/reparsing-cooked-data-coming-from-a-heavy-forwarder-possibl... . Then you setup your transforms for overriding index name on third party Splunk instance. https://answers.splunk.com/answers/246672/how-can-i-override-an-index-name-based-on-sourcety.html

0 Karma

meidal_splunkin
New Member

thanks a lot for the help. I managed to get it working

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...