topic Re: How to send a data source to specific indexers? in Getting Data In

How to send a data source to specific indexers?

Glasses — Thu, 24 Sep 2020 13:45:55 GMT

Hi I am looking for an example to follow, where I can specify which data source goes to which indexers.

I am trying to cutover data sources from my HF tier to a new indexer cluster.

Currently all my HFs output to my existing indexers (unclustered).

1st>>> I want to try sending data from a tcp listener port (e.g. tcp1234 local on a single HF) to my new indexer cluster only. But I want the other data sources flowing thru the HF to go to the existing indexers.

2nd>>> I want to try changing the UFs on some windows hosts to send to the HF tier (but configure the HFs) and then send to the new indexers.

Is _tcp_routing the way to go on the inputs?

Any advice appreciated. Thank you.

Re: How to send a data source to specific indexers?

thambisetty — Thu, 24 Sep 2020 13:49:43 GMT

@Glasses

check below thread:

https://community.splunk.com/t5/Getting-Data-In/How-can-I-route-data-to-specific-indexers-using-a-heavy/m-p/225200

Re: How to send a data source to specific indexers?

Glasses — Thu, 24 Sep 2020 14:16:23 GMT

thank you, I will try this, I am just confused by all the different ways this is described...

Re: How to send a data source to specific indexers?

Glasses — Thu, 24 Sep 2020 16:22:08 GMT

So after reading the splunk docs, here is what I think is what I need to do for a tcp listener.

inputs.conf (located in .../apps)

[tcp://1111]
connection_host = dns
index = foo
sourcetype = bar

Props.conf (located in .../apps)

[bar]
TRANSFORMS-routing=New_IDX_CLUSTER_Routing

Transforms.conf (located in .../apps)

[New_IDX_CLUSTER_Routing]
DEST_KEY=_TCP_ROUTING
Format=NewClusterGroup

Outputs.conf (located in .../system/local)

(other defaults stuff in here)

[tcpout:NewClusterGroup]
server=10.10.10.10:9997, 10.10.10.11:9997, 10.10.10.12:9997

Does this look right?

Re: How to send a data source to specific indexers?

isoutamo — Thu, 24 Sep 2020 18:27:45 GMT

When you have inputs.conf with tcp listener you could define _TCP_ROUTING = <tcpout_group_name>,<tcpout_group_name>
directly there, no need to do it with props and transforms.
Check it from https://docs.splunk.com/Documentation/Splunk/8.0.6/Admin/Inputsconf

Wit those UFs you should use splunktcp receiver as you already use on indexers. And remember that you couldn’t do any props & transforms stuff on indexers after events are going through HF.

r. Ismo

Re: How to send a data source to specific indexers?

Glasses — Thu, 24 Sep 2020 20:05:35 GMT

so then like this?

Inputs.conf

[tcp://1111]
connection_host = dns
index = foo
sourcetype = bar
_TCP_ROUTING = NewClusterGroup

outputs.conf

[tcpout:NewClusterGroup]
server=10.10.10.10:9997, 10.10.10.11:9997, 10.10.10.12:9997

Re: How to send a data source to specific indexers?

Glasses — Thu, 24 Sep 2020 20:32:55 GMT

yes that is the way to go!!!

adding _TCP_ROUTING to the inputs.com that is ...

that works, while the other previously posted solution with props and transforms didn't work for me (I musta mucked something up)