Getting Data In

How to redirect some data coming into an indexer (HEC) to another indexer?

twinspop
Influencer

I have Http Event Collector inputs defined on an indexer cluster. I need to send one of the tokens' data to a different indexer. _TCP_ROUTING in inputs, plus an outputs.conf def?
If so, what magic in outputs.conf do I need to ensure most traffic ignores the special case and just indexes normally?

0 Karma
1 Solution

twinspop
Influencer

The bottom of this page has an example of how to do it using selective indexing.

https://docs.splunk.com/Documentation/Splunk/7.3.0/Admin/Outputsconf

View solution in original post

0 Karma

twinspop
Influencer

The bottom of this page has an example of how to do it using selective indexing.

https://docs.splunk.com/Documentation/Splunk/7.3.0/Admin/Outputsconf

0 Karma

jkat54
SplunkTrust
SplunkTrust

Yes, your proposed method will work. I've done it before just fine.

Inputs:

[yourstanza]
_TCP_ROUTING=YourRoutingGroup

Outputs:

[splunk-tcp://YourRoutingGroup]
server=yourserver

Everything else will use the default routing group

Here's an example using plain TCP:

[tcpout]
defaultGroup=everythingElseGroup

[tcpout:syslogGroup]
server=10.1.1.197:9996, 10.1.1.198:9997

[tcpout:errorGroup]
server=10.1.1.200:9999

[tcpout:everythingElseGroup]
server=10.1.1.250:6666

0 Karma

twinspop
Influencer

That didn't work. I added this stanza (alone) to the CM and applied. No other changes. I had assumed that default would remain undefined and therefore it would index locally.

[tcpout:dc1_indexers]
server = dc1_indexers:9997
autoLBFrequency = 20
autoLBVolume = 10000
compressed = true
useACK = false

All locally indexed data disappeared, and tons of logs regarding TcpOutputProc connections to the indexers in the dc1_indexers cluster above.

So how do you add an output destination that will not take over default when you want to maintain local indexing?

0 Karma

jkat54
SplunkTrust
SplunkTrust

You can also use regex in transforms to set the tcp routing:

https://docs.splunk.com/Documentation/Splunk/7.3.0/Forwarding/Routeandfilterdatad

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Unlocking Unified Insights: New Gigamon Federated Search App for Splunk

In today’s data-heavy environment, organizations are caught in a data distribution dilemma. As data volumes ...

GA: New Data Management App in Splunk Platform

Streamlining Data Management: Introducing a unified experience in Splunk Managing data at scale shouldn’t feel ...

Announcing Modern Navigation: A New Era of Splunk User Experience

We are excited to introduce the Modern Navigation feature in the Splunk Platform, available to both cloud and ...