Hi at all,
I have some Heavy Forwarders that receive data from some Universal Forwarders and take syslogs from some appliances.
HFs take these logs and forward all of them to an Indexer (Indexer_A) and all the flow runs correctly.
My problem is that I have to forward a part of these logs to another Splunk (Indexer_B) (I know that these logs are indexed two times!):
I configured Indexer_A as described in https://docs.splunk.com/Documentation/Splunk/8.0.1/Forwarding/Routeandfilterdatad but I receive in Indexer_B all the data of Indexer_A, also Splunk internal logs.
These are my configurations on Indexer_A
outputs.conf
[tcpout]
indexAndForward = 1
[tcpout:Indexer_B]
disabled = 0
server = 1.1.1.1:9997
props.conf
[WinEventLog:Security]
TRANSFORMS-routing=fw_win
transforms.conf
[fw_win]
REGEX=.
DEST_KEY=_TCP_ROUTING
FORMAT=Indexer_B
I'd like to forward only WinEventLog:Security logs but I receinve also other logs, also Splunk internal logs.
My question is: if Indexer_A receives logs from an Heavy Forwarder is it able to forward a subset of data to another Indexer or must I do this on HFs?
If it's possible what I'm doing wrong?
Ciao and thank you.
Giuseppe
Based from my analysis, it's not working because the data is already cooked when it reaches indexer_A, so those props/transforms do nothing. Those props/transforms aren't executed on indexer_a at all, because the HF have parsed the data already, sent it to indexer_a as "cooked data", and therefore indexer_a will just put them to disk and skip it's local props/transforms.
Wanted to confirm jarizeloyola answer, you need to do this at the source enterprise server or not pass the data through a HF at all!
You can re-cook data but it is unsupported and would not be a good idea on the indexing tier...
You might be able to use CLONE_SOURCETYPE at the HF and then route it with changes refer to:
https://answers.splunk.com/answers/556300/how-can-i-use-clone-sourcetype-to-send-a-cloned-mo.html
Based from my analysis, it's not working because the data is already cooked when it reaches indexer_A, so those props/transforms do nothing. Those props/transforms aren't executed on indexer_a at all, because the HF have parsed the data already, sent it to indexer_a as "cooked data", and therefore indexer_a will just put them to disk and skip it's local props/transforms.
Hi @jarizeloyola,
thank you very much, for you help!
it's a liitation!.
Ciao and next time.
Giuseppe