Getting Data In

How to redirect some data coming into an indexer (HEC) to another indexer?

twinspop
Influencer

I have Http Event Collector inputs defined on an indexer cluster. I need to send one of the tokens' data to a different indexer. _TCP_ROUTING in inputs, plus an outputs.conf def?
If so, what magic in outputs.conf do I need to ensure most traffic ignores the special case and just indexes normally?

0 Karma
1 Solution

twinspop
Influencer

The bottom of this page has an example of how to do it using selective indexing.

https://docs.splunk.com/Documentation/Splunk/7.3.0/Admin/Outputsconf

View solution in original post

0 Karma

twinspop
Influencer

The bottom of this page has an example of how to do it using selective indexing.

https://docs.splunk.com/Documentation/Splunk/7.3.0/Admin/Outputsconf

View solution in original post

0 Karma

jkat54
SplunkTrust
SplunkTrust

Yes, your proposed method will work. I've done it before just fine.

Inputs:

[yourstanza]
_TCP_ROUTING=YourRoutingGroup

Outputs:

[splunk-tcp://YourRoutingGroup]
server=yourserver

Everything else will use the default routing group

Here's an example using plain TCP:

[tcpout]
defaultGroup=everythingElseGroup

[tcpout:syslogGroup]
server=10.1.1.197:9996, 10.1.1.198:9997

[tcpout:errorGroup]
server=10.1.1.200:9999

[tcpout:everythingElseGroup]
server=10.1.1.250:6666

0 Karma

twinspop
Influencer

That didn't work. I added this stanza (alone) to the CM and applied. No other changes. I had assumed that default would remain undefined and therefore it would index locally.

[tcpout:dc1_indexers]
server = dc1_indexers:9997
autoLBFrequency = 20
autoLBVolume = 10000
compressed = true
useACK = false

All locally indexed data disappeared, and tons of logs regarding TcpOutputProc connections to the indexers in the dc1_indexers cluster above.

So how do you add an output destination that will not take over default when you want to maintain local indexing?

0 Karma

jkat54
SplunkTrust
SplunkTrust

You can also use regex in transforms to set the tcp routing:

https://docs.splunk.com/Documentation/Splunk/7.3.0/Forwarding/Routeandfilterdatad

0 Karma
Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!