Getting Data In

If an indexer receives logs from a heavy forwarder is it able to forward a subset of data to another Indexer?

Legend

Hi at all,
I have some Heavy Forwarders that receive data from some Universal Forwarders and take syslogs from some appliances.
HFs take these logs and forward all of them to an Indexer (Indexer_A) and all the flow runs correctly.

My problem is that I have to forward a part of these logs to another Splunk (IndexerB) (I know that these logs are indexed two times!):
I configured Indexer
A as described in https://docs.splunk.com/Documentation/Splunk/8.0.1/Forwarding/Routeandfilterdatad but I receive in IndexerB all the data of IndexerA, also Splunk internal logs.
These are my configurations on Indexer_A

outputs.conf

[tcpout]
indexAndForward = 1

[tcpout:Indexer_B]
disabled = 0
server = 1.1.1.1:9997

props.conf

[WinEventLog:Security]
TRANSFORMS-routing=fw_win

transforms.conf

[fw_win]
REGEX=.
DEST_KEY=_TCP_ROUTING
FORMAT=Indexer_B

I'd like to forward only WinEventLog:Security logs but I receinve also other logs, also Splunk internal logs.

My question is: if Indexer_A receives logs from an Heavy Forwarder is it able to forward a subset of data to another Indexer or must I do this on HFs?
If it's possible what I'm doing wrong?

Ciao and thank you.
Giuseppe

0 Karma
1 Solution

Path Finder

Based from my analysis, it's not working because the data is already cooked when it reaches indexerA, so those props/transforms do nothing. Those props/transforms aren't executed on indexera at all, because the HF have parsed the data already, sent it to indexera as "cooked data", and therefore indexera will just put them to disk and skip it's local props/transforms.

View solution in original post

0 Karma

SplunkTrust
SplunkTrust

Wanted to confirm jarizeloyola answer, you need to do this at the source enterprise server or not pass the data through a HF at all!

You can re-cook data but it is unsupported and would not be a good idea on the indexing tier...

You might be able to use CLONE_SOURCETYPE at the HF and then route it with changes refer to:
https://answers.splunk.com/answers/556300/how-can-i-use-clone-sourcetype-to-send-a-cloned-mo.html

Path Finder

Based from my analysis, it's not working because the data is already cooked when it reaches indexerA, so those props/transforms do nothing. Those props/transforms aren't executed on indexera at all, because the HF have parsed the data already, sent it to indexera as "cooked data", and therefore indexera will just put them to disk and skip it's local props/transforms.

View solution in original post

0 Karma

Legend

Hi @jarizeloyola,
thank you very much, for you help!
it's a liitation!.

Ciao and next time.
Giuseppe

0 Karma