Getting Data In

How to send a data source to specific indexers?

Glasses
Builder

Hi I am looking for an example to follow, where I can specify which data source goes to which indexers.

I am trying to cutover data sources from my HF tier to a new indexer cluster.

Currently all my HFs output to my existing indexers (unclustered).

1st>>> I want to try sending data from a tcp listener port (e.g. tcp1234 local on a single HF) to my new indexer cluster only.  But I want the other data sources flowing thru the HF to go to the existing indexers.

2nd>>> I want to try changing the UFs on some windows hosts to send to the HF tier (but configure the HFs) and then send to the new indexers.

Is _tcp_routing the way to go on the inputs?

 

Any advice appreciated.  Thank you.

1 Solution

isoutamo
SplunkTrust
SplunkTrust

When you have inputs.conf with tcp listener you could define _TCP_ROUTING = <tcpout_group_name>,<tcpout_group_name>
directly there, no need to do it with props and transforms.
Check it from https://docs.splunk.com/Documentation/Splunk/8.0.6/Admin/Inputsconf

Wit those UFs you should use splunktcp receiver as you already use on indexers. And remember that you couldn’t do any props & transforms stuff on indexers after events are going through HF.

r. Ismo

View solution in original post

Glasses
Builder

thank you, I will try this, I am just confused by all the different ways this is described...

0 Karma

thambisetty
SplunkTrust
SplunkTrust

@Glasses 

check below thread:

https://community.splunk.com/t5/Getting-Data-In/How-can-I-route-data-to-specific-indexers-using-a-he...

————————————
If this helps, give a like below.

Glasses
Builder

So after reading the splunk docs, here is what I think  is what I need to do for a tcp listener.

 

inputs.conf (located in .../apps) 


[tcp://1111]
connection_host = dns
index = foo
sourcetype = bar

 

Props.conf (located in .../apps)

[bar]
TRANSFORMS-routing=New_IDX_CLUSTER_Routing

 

 

Transforms.conf (located in .../apps)

[New_IDX_CLUSTER_Routing]
DEST_KEY=_TCP_ROUTING
Format=NewClusterGroup

 

 

Outputs.conf (located in .../system/local)

(other defaults stuff in here)

[tcpout:NewClusterGroup]
server=10.10.10.10:9997, 10.10.10.11:9997, 10.10.10.12:9997

 

Does this look right?

 

isoutamo
SplunkTrust
SplunkTrust

When you have inputs.conf with tcp listener you could define _TCP_ROUTING = <tcpout_group_name>,<tcpout_group_name>
directly there, no need to do it with props and transforms.
Check it from https://docs.splunk.com/Documentation/Splunk/8.0.6/Admin/Inputsconf

Wit those UFs you should use splunktcp receiver as you already use on indexers. And remember that you couldn’t do any props & transforms stuff on indexers after events are going through HF.

r. Ismo

Glasses
Builder

so then like this?

Inputs.conf

[tcp://1111]
connection_host = dns
index = foo
sourcetype = bar
_TCP_ROUTING = NewClusterGroup

 

outputs.conf 

[tcpout:NewClusterGroup]
server=10.10.10.10:9997, 10.10.10.11:9997, 10.10.10.12:9997

Glasses
Builder

yes that is the way to go!!!

adding  _TCP_ROUTING to the inputs.com that is ...

that works, while the other previously posted solution with props and transforms didn't work for me (I musta mucked something up)

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...