Getting Data In

How to send a data source to specific indexers?

Glasses
Builder

Hi I am looking for an example to follow, where I can specify which data source goes to which indexers.

I am trying to cutover data sources from my HF tier to a new indexer cluster.

Currently all my HFs output to my existing indexers (unclustered).

1st>>> I want to try sending data from a tcp listener port (e.g. tcp1234 local on a single HF) to my new indexer cluster only.  But I want the other data sources flowing thru the HF to go to the existing indexers.

2nd>>> I want to try changing the UFs on some windows hosts to send to the HF tier (but configure the HFs) and then send to the new indexers.

Is _tcp_routing the way to go on the inputs?

 

Any advice appreciated.  Thank you.

1 Solution

isoutamo
SplunkTrust
SplunkTrust

When you have inputs.conf with tcp listener you could define _TCP_ROUTING = <tcpout_group_name>,<tcpout_group_name>
directly there, no need to do it with props and transforms.
Check it from https://docs.splunk.com/Documentation/Splunk/8.0.6/Admin/Inputsconf

Wit those UFs you should use splunktcp receiver as you already use on indexers. And remember that you couldn’t do any props & transforms stuff on indexers after events are going through HF.

r. Ismo

View solution in original post

Glasses
Builder

thank you, I will try this, I am just confused by all the different ways this is described...

0 Karma

thambisetty
SplunkTrust
SplunkTrust

@Glasses 

check below thread:

https://community.splunk.com/t5/Getting-Data-In/How-can-I-route-data-to-specific-indexers-using-a-he...

————————————
If this helps, give a like below.

Glasses
Builder

So after reading the splunk docs, here is what I think  is what I need to do for a tcp listener.

 

inputs.conf (located in .../apps) 


[tcp://1111]
connection_host = dns
index = foo
sourcetype = bar

 

Props.conf (located in .../apps)

[bar]
TRANSFORMS-routing=New_IDX_CLUSTER_Routing

 

 

Transforms.conf (located in .../apps)

[New_IDX_CLUSTER_Routing]
DEST_KEY=_TCP_ROUTING
Format=NewClusterGroup

 

 

Outputs.conf (located in .../system/local)

(other defaults stuff in here)

[tcpout:NewClusterGroup]
server=10.10.10.10:9997, 10.10.10.11:9997, 10.10.10.12:9997

 

Does this look right?

 

isoutamo
SplunkTrust
SplunkTrust

When you have inputs.conf with tcp listener you could define _TCP_ROUTING = <tcpout_group_name>,<tcpout_group_name>
directly there, no need to do it with props and transforms.
Check it from https://docs.splunk.com/Documentation/Splunk/8.0.6/Admin/Inputsconf

Wit those UFs you should use splunktcp receiver as you already use on indexers. And remember that you couldn’t do any props & transforms stuff on indexers after events are going through HF.

r. Ismo

Glasses
Builder

so then like this?

Inputs.conf

[tcp://1111]
connection_host = dns
index = foo
sourcetype = bar
_TCP_ROUTING = NewClusterGroup

 

outputs.conf 

[tcpout:NewClusterGroup]
server=10.10.10.10:9997, 10.10.10.11:9997, 10.10.10.12:9997

Glasses
Builder

yes that is the way to go!!!

adding  _TCP_ROUTING to the inputs.com that is ...

that works, while the other previously posted solution with props and transforms didn't work for me (I musta mucked something up)

Get Updates on the Splunk Community!

AppDynamics Summer Webinars

This summer, our mighty AppDynamics team is cooking up some delicious content on YouTube Live to satiate your ...

SOCin’ it to you at Splunk University

Splunk University is expanding its instructor-led learning portfolio with dedicated Security tracks at .conf25 ...

Credit Card Data Protection & PCI Compliance with Splunk Edge Processor

Organizations handling credit card transactions know that PCI DSS compliance is both critical and complex. The ...