Splunk Search

How to query _TCP_ROUTING key value?

jcspigler2010
Path Finder

Hello all

Is there a way you can query event's _TCP_ROUTING key value in a search? I would assume you should be able to since its being written in the events meta. We have bit of a complicated data routing and filtering issue and a search like this would be super helpful.

Thanks!

0 Karma
1 Solution

somesoni2
Revered Legend

I don't believe the _TCP_ROUTING is written into metadata for each event. It's an attribute used by Splunk engine (of heavy forwarder/indexer) to route events to different queue/ tcpout groups, it's not assigned to event.

View solution in original post

somesoni2
Revered Legend

I don't believe the _TCP_ROUTING is written into metadata for each event. It's an attribute used by Splunk engine (of heavy forwarder/indexer) to route events to different queue/ tcpout groups, it's not assigned to event.

View solution in original post

s2_splunk
Splunk Employee
Splunk Employee

Correct, hence converted to answer for your convenience. 🙂

jcspigler2010
Path Finder

Thanks somesoni2!

Sorry for the delay in credit. Was onsite with a customer all day. If you would be so kind to humor my next question. I can always start another thread if need be.

But we have an odd requirement at a client site where they index EVERYTHING sent from a heavy forwarder to an indexer and then send 2 or 3 source types off to a remote indexer. Now I know this isn't the conventional or best practice way of doing stuff, but due to political and environment boundaries, they have to forward data from the indexer and not the HF. HF would obviously be the optimal place to do this. So my question is, can we set the _TCP_ROUTING key at the HF level, and use it for routing and forwarding decisions at the indexer level? I guess the question generalized, does the DEST_KEY and FORMAT values persist from splunk instance to splunk instance or is just internal to that instance. I am going to test this out this week but figured someone would already have the answer.

Thanks!

0 Karma

somesoni2
Revered Legend

The routing transformation are applied during cooking/parsing of data, which will be done at Heavy forwarder level in your use case. Once cooked/parsed at HF level, it won't be parsed again (default and understandably) at indexer level. Now you can configure Indexers to re-parse the data but a) not recommended, b) it can't be done for a single sourcetype, but will be done for all data coming to indexers. So, basically you'll be doing double processing, consuming double resources and double latency for parsing. If that's something you can live with, see this on how to implement it.

https://answers.splunk.com/answers/97918/reparsing-cooked-data-coming-from-a-heavy-forwarder-possibl...
https://answers.splunk.com/answers/224312/hf1-hf2-indexer-how-to-route-a-set-of-data-that-ha.html

One more thing, you'll need to ensure that your Indexers have all the props.conf/transforms.conf that are in Heavy forwarder (because of re-parsing).

s2_splunk
Splunk Employee
Splunk Employee

Hmmmm, while semantically correct, I don't think that's applicable. You should be able to affect TCP routing wherever you have an outputs.conf. Functionally, an indexer is nothing other than a heavy forwarder that writes to disk, so you should be able to that there as well.

anand_singh17
Path Finder

But after lot trials, not able to forward data from indexers.

0 Karma

s2_splunk
Splunk Employee
Splunk Employee

Are you sure you followed the documentation on the topic and placed the appropriate configuration files in the appropriate places?

IndexAndForward should work just fine on an indexer...?

0 Karma

anand_singh17
Path Finder

Hi Ssievert,

We do not have HF, where we can apply these configs.

We have 500 devices and have 98% of them sending data from UF, so as per optimization, we did not go for HF.

When applying the settings, UF is not able to process them.

0 Karma

jcspigler2010
Path Finder

Thanks somesoni2

0 Karma