topic Re: Data Routing through multiple HF in Getting Data In

How to route data with props and transforms over multiple HF?

CMEOGNAD — Wed, 18 Jan 2023 17:56:34 GMT

Hi Community,

how to route data with props and transforms over multiple HF?

Source A to Data Collector > IDX Cluster A
|
(Data Copy A)
|
|---> Source B to Data Collector > IDX Cluster A/B

Currently, the routing only works directly to IDX Cluster A/B, but not via Source B HF

Please Help - Markus

Re: Data Routing through multiple HF

gcusello — Tue, 17 Jan 2023 12:54:55 GMT

Hi @CMEOGNAD,

lete me understand: are Source A and SourceB different systems with Universal Forwarders or different Data Source on the same Universal Forwarder, or what else?

If they are different Universal Forwarders you can use a different outputs.con on each one.

If they are different Data Sources on the same Universal Forwarder, you have to follow the instructions at https://docs.splunk.com/Documentation/Splunk/9.0.3/Forwarding/Routeandfilterdatad#Route_inputs_to_specific_indexers_based_on_the_data_input

Ciao.

Giuseppe

Re: Data Routing through multiple HF

CMEOGNAD — Tue, 17 Jan 2023 14:00:32 GMT

"Source A" to Data Collector 01 > IDX Cluster 01 for Data A
|
(Data Copy from Source A for Cluster 02)
|
|---> "Source B" to Data Collector 02 > IDX Cluster 02 for Data A and B

THX for the link, my config is similar, but... the "Data A" Copy from DC01 direct to IDX Cluster 02 works fine, "Data A" Copy from DC01 to DC02 to IDX Cluster 02 not...

Any idea?
Is "Data HF hopping" 😉 even possible?
Without transforms and props directly via tcp-out over inputs.conf it worked...

Re: Data Routing through multiple HF

gcusello — Tue, 17 Jan 2023 14:07:33 GMT

Hi @CMEOGNAD,

if DataA is a different Universal Forwarder, you could directly send data to both the DCs not passing throgh DC1.

You can configure in outputs.conf (as described in the documentation) a "fork" to send data to both the destinations.

Ciao.

Giuseppe

Re: Data Routing through multiple HF

PickleRick — Tue, 17 Jan 2023 15:42:40 GMT

If you want to have your data processed by HFA then sent to HFB where it will again be processed by props/transforms and then sent to indexers... That can't be done unless you do something ugly in between like sending HFA->HFB via syslog or HEC. Your data stream is parsed at the first parsing Splunk instance in event's path. Then it's passed as parsed data and not touched any more.