topic Re: Forwarde data based on sourcetype between 2 indexers or between indexer and search head in Splunk Enterprise

Forwarde data based on sourcetype between 2 indexers or between indexer and search head

myitlab1000 — Wed, 22 Jul 2020 15:55:57 GMT

Hello,

Il would like to know if i could forward data based on sourcetype between 2 indexers or between indexer and search head.

Il would like to forward only data of a certain sourcetype.

Thank you for your help

Re: Forwarde data based on sourcetype between 2 indexers or between indexer and search head

richgalloway — Wed, 22 Jul 2020 17:25:43 GMT

Forwarding between indexers is possible. Forwarding from indexer to search head does not make sense since search heads do not store data.
What problem are you trying to solve?

Re: Forwarde data based on sourcetype between 2 indexers or between indexer and search head

myitlab1000 — Wed, 22 Jul 2020 18:01:57 GMT

I have multiple indexers and one search head.

forwarders => Indexer 1, Indexer 2, Indexer N => search head => forwarding to third party

I can forward data but the problem is that is forwarding all the data.

Il would like to index all data locally to indexer and forward only data based on certain sourcetype by the search head to avoid open additional port between indexers and the third party software.

I have tested by configuring props.conf, transforms.conf and outputs.conf, but still forwarding all data, all sourcetype.

reference docs : https://docs.splunk.com/Documentation/Splunk/8.0.5/Forwarding/Routeandfilterdatad

Thanks a lot for your help

Re: Forwarde data based on sourcetype between 2 indexers or between indexer and search head

richgalloway — Wed, 22 Jul 2020 19:15:13 GMT

How are you specifying the sourcetype to forward?

Re: Forwarde data based on sourcetype between 2 indexers or between indexer and search head

myitlab1000 — Thu, 23 Jul 2020 07:26:41 GMT

Here is my conf of an indexer to forward to search head and from search i would like to forward to third party.

The problem is not only data of soucetype "mysourcetype" is forwarded but all data.

in props.conf:

[mysourcetype]

TRANSFORMS-routing = forward_to_my_search_head_from_indexer

in transforms.conf:

[forward_to_my_search_head_from_indexer]
REGEX = .
DEST_KEY = _TCP_ROUTING
FORMAT = my_search_head_group

in outpus.conf:

[tcpout]
defaultGroup = nothing
indexAndForward = true

[tcpout:my_search_head_group]
disable = false
server = my_search_head_ip:9997
sendCookedData = false

Thank you for yo

Re: Forwarde data based on sourcetype between 2 indexers or between indexer and search head

richgalloway — Thu, 23 Jul 2020 15:00:10 GMT

Your configuration appears to make sense according to the documentation, however, I still cannot wrap my head around why you are forwarding data from an indexer to a search head. That is not a normal practice. Can you describe your Splunk architecture? What problem are you trying to solve by forwarding data from indexer to SH?

Re: Forwarde data based on sourcetype between 2 indexers or between indexer and search head

myitlab1000 — Thu, 23 Jul 2020 18:27:36 GMT

I would like to expose one port from SH to external (third party software).

Re: Forwarde data based on sourcetype between 2 indexers or between indexer and search head

richgalloway — Thu, 23 Jul 2020 20:38:57 GMT

Please say more about that. Why the SH and not the indexer where the data resides? What third-party software)?
I think your defaultGroup attribute needs a value that is not "my_search_head_group".
Have you read https://docs.splunk.com/Documentation/Splunk/8.0.5/Forwarding/Forwarddatatothird-partysystemsd?