Getting Data In

If an indexer receives logs from a heavy forwarder is it able to forward a subset of data to another Indexer?

gcusello
SplunkTrust
SplunkTrust

Hi at all,
I have some Heavy Forwarders that receive data from some Universal Forwarders and take syslogs from some appliances.
HFs take these logs and forward all of them to an Indexer (Indexer_A) and all the flow runs correctly.

My problem is that I have to forward a part of these logs to another Splunk (Indexer_B) (I know that these logs are indexed two times!):
I configured Indexer_A as described in https://docs.splunk.com/Documentation/Splunk/8.0.1/Forwarding/Routeandfilterdatad but I receive in Indexer_B all the data of Indexer_A, also Splunk internal logs.
These are my configurations on Indexer_A

outputs.conf

[tcpout]
indexAndForward = 1

[tcpout:Indexer_B]
disabled = 0
server = 1.1.1.1:9997

props.conf

[WinEventLog:Security]
TRANSFORMS-routing=fw_win

transforms.conf

[fw_win]
REGEX=.
DEST_KEY=_TCP_ROUTING
FORMAT=Indexer_B

I'd like to forward only WinEventLog:Security logs but I receinve also other logs, also Splunk internal logs.

My question is: if Indexer_A receives logs from an Heavy Forwarder is it able to forward a subset of data to another Indexer or must I do this on HFs?
If it's possible what I'm doing wrong?

Ciao and thank you.
Giuseppe

0 Karma
1 Solution

jarizeloyola
Path Finder

Based from my analysis, it's not working because the data is already cooked when it reaches indexer_A, so those props/transforms do nothing. Those props/transforms aren't executed on indexer_a at all, because the HF have parsed the data already, sent it to indexer_a as "cooked data", and therefore indexer_a will just put them to disk and skip it's local props/transforms.

View solution in original post

0 Karma

gjanders
SplunkTrust
SplunkTrust

Wanted to confirm jarizeloyola answer, you need to do this at the source enterprise server or not pass the data through a HF at all!

You can re-cook data but it is unsupported and would not be a good idea on the indexing tier...

You might be able to use CLONE_SOURCETYPE at the HF and then route it with changes refer to:
https://answers.splunk.com/answers/556300/how-can-i-use-clone-sourcetype-to-send-a-cloned-mo.html

jarizeloyola
Path Finder

Based from my analysis, it's not working because the data is already cooked when it reaches indexer_A, so those props/transforms do nothing. Those props/transforms aren't executed on indexer_a at all, because the HF have parsed the data already, sent it to indexer_a as "cooked data", and therefore indexer_a will just put them to disk and skip it's local props/transforms.

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @jarizeloyola,
thank you very much, for you help!
it's a liitation!.

Ciao and next time.
Giuseppe

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...