topic Re: What is the difference between DEST_KEY= _TCP_ROUTING and DEST_KEY = _MetaData:Index in Getting Data In

What is the difference between DEST_KEY= _TCP_ROUTING and DEST_KEY = _MetaData:Index

manikanta66 — Tue, 05 Dec 2017 11:36:35 GMT

Please give me a practical explanation of DEST_KEY usage in transforms.conf

Re: What is the difference between DEST_KEY= _TCP_ROUTING and DEST_KEY = _MetaData:Index

harsmarvania57 — Tue, 29 Sep 2020 17:04:11 GMT

_TCP_ROUTING routes set of data to set of servers which you have defined in outputs.conf with stanza starting from [tcpout:....]
_MetaData:Index will route set of data to different index.

Example: _TCP_ROUTING

props.conf

[test]
TRANSFORMS-routing=errorRouting

transforms.conf

[errorRouting]
REGEX=error
DEST_KEY=_TCP_ROUTING
FORMAT=errorGroup

outputs.conf

[tcpout:errorGroup]
server=10.10.0.1:9997, 10.10.0.2:9997

In above example splunk will find error word from test sourcetype raw data and it will send it to tcpout group errorGroup which will send data to 2 indexers in load balanced way.

Example: _MetaData:Index

props.conf

[test]
TRANSFORMS-routing=errorRouting

transforms.conf

[errorRouting]
REGEX=error
DEST_KEY=_MetaData:Index
FORMAT= error_index

Let's assume that test sourcetype is sending data to ok_index index, now based on above example splunk will find raw data from test sourcetype with word error and it will write it to error_index index.

I hope this helps.

Thanks,
Harshil

Re: What is the difference between DEST_KEY= _TCP_ROUTING and DEST_KEY = _MetaData:Index

manikanta66 — Wed, 06 Dec 2017 12:49:48 GMT

Hi @harsmarvania57,

Thanks for your response. I have a doubt in your answer.

as per your 2nd example, the UF contain inputs.conf like below?

[monitor://var/www/testing.log]
disabled = 0
sourcetype = test
index = ok_index

you said "test sourcetype with word error will write it to error_index"
for the FORMAT key value, you mention error_index

here my doubts are
1.which name I choose for a new index in the indexer
ok index OR error_index?
2.If I have two indexers, How HF will find index without specifying the target-group in the FORMAT?

Re: What is the difference between DEST_KEY= _TCP_ROUTING and DEST_KEY = _MetaData:Index

harsmarvania57 — Wed, 06 Dec 2017 13:05:19 GMT

Based on 2nd example both index will require on your Indexers (ok_index and error_index)

Let's say you have below events in your log files

30-10-2017GMT17:12:00 ERROR This is error
30-10-2017GMT17:12:50 INFO This is info

In this case 1st line(event) will got to error_index and 2nd line(event) will go to ok_index

You need to specify index name in FORMAT on HF in 2nd example, otherwise it will not work.

Re: What is the difference between DEST_KEY= _TCP_ROUTING and DEST_KEY = _MetaData:Index

manikanta66 — Thu, 07 Dec 2017 05:48:24 GMT

Thank you @harsmarvania57