Getting Data In

What is the difference between DEST_KEY= _TCP_ROUTING and DEST_KEY = _MetaData:Index

Explorer

Please give me a practical explanation of DEST_KEY usage in transforms.conf

0 Karma
1 Solution

SplunkTrust
SplunkTrust

Hi @manikanta66,

_TCP_ROUTING routes set of data to set of servers which you have defined in outputs.conf with stanza starting from [tcpout:....]
_MetaData:Index will route set of data to different index.

Example: _TCP_ROUTING

props.conf

[test]
TRANSFORMS-routing=errorRouting

transforms.conf

[errorRouting]
REGEX=error
DEST_KEY=_TCP_ROUTING
FORMAT=errorGroup

outputs.conf

[tcpout:errorGroup]
server=10.10.0.1:9997, 10.10.0.2:9997

In above example splunk will find error word from test sourcetype raw data and it will send it to tcpout group errorGroup which will send data to 2 indexers in load balanced way.

Example: _MetaData:Index

props.conf

[test]
TRANSFORMS-routing=errorRouting

transforms.conf

[errorRouting]
REGEX=error
DEST_KEY=_MetaData:Index
FORMAT= error_index

Let's assume that test sourcetype is sending data to ok_index index, now based on above example splunk will find raw data from test sourcetype with word error and it will write it to error_index index.

I hope this helps.

Thanks,
Harshil

View solution in original post

SplunkTrust
SplunkTrust

Hi @manikanta66,

_TCP_ROUTING routes set of data to set of servers which you have defined in outputs.conf with stanza starting from [tcpout:....]
_MetaData:Index will route set of data to different index.

Example: _TCP_ROUTING

props.conf

[test]
TRANSFORMS-routing=errorRouting

transforms.conf

[errorRouting]
REGEX=error
DEST_KEY=_TCP_ROUTING
FORMAT=errorGroup

outputs.conf

[tcpout:errorGroup]
server=10.10.0.1:9997, 10.10.0.2:9997

In above example splunk will find error word from test sourcetype raw data and it will send it to tcpout group errorGroup which will send data to 2 indexers in load balanced way.

Example: _MetaData:Index

props.conf

[test]
TRANSFORMS-routing=errorRouting

transforms.conf

[errorRouting]
REGEX=error
DEST_KEY=_MetaData:Index
FORMAT= error_index

Let's assume that test sourcetype is sending data to ok_index index, now based on above example splunk will find raw data from test sourcetype with word error and it will write it to error_index index.

I hope this helps.

Thanks,
Harshil

View solution in original post

Explorer

Hi @harsmarvania57,

Thanks for your response. I have a doubt in your answer.

as per your 2nd example, the UF contain inputs.conf like below?

  1. [monitor://var/www/testing.log]
  2. disabled = 0
  3. sourcetype = test
  4. index = ok_index

you said "test sourcetype with word error will write it to error_index"
for the FORMAT key value, you mention error_index

here my doubts are
1.which name I choose for a new index in the indexer
ok index OR error_index?
2.If I have two indexers, How HF will find index without specifying the target-group in the FORMAT?

0 Karma

SplunkTrust
SplunkTrust

Based on 2nd example both index will require on your Indexers (ok_index and error_index)

Let's say you have below events in your log files

30-10-2017GMT17:12:00 ERROR This is error
30-10-2017GMT17:12:50 INFO This is info

In this case 1st line(event) will got to error_index and 2nd line(event) will go to ok_index

You need to specify index name in FORMAT on HF in 2nd example, otherwise it will not work.

0 Karma

Explorer

Thank you @harsmarvania57

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!