How to filter data from a single file and write to two different indexes?

[payal23]

I am trying to filter a set of data from a single file with the below conditions and send the filtered data to different indexes.

Events are like: [ file.txt]

<85>XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

<25>XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

So, event with 85 one should go to index A

and 25 one should go to index B

[thambisetty · Accepted solution]

create props.conf On HF or Indexer

Note: you should change [currentsourcetype] below with your sourcetype events are coming in.

[currentsourcetype]
TRANSFORMS-routetoindexes = firstindex, secondindex

transforms.conf on same HF or Indexers

[firstindex]
REGEX = ^\<85\> 
#to change index if event matches with the above regex
DEST_KEY = _MetaData:Index 
#indexA is index where matching data will be indexed
FORMAT = indexA

[secondindex]
REGEX = ^\<25\>
#to change index if event matches with the above regex
DEST_KEY = _MetaData:Index 
#indexA is index where matching data will be indexed
FORMAT = indexB

[payal23]

Thanks @thambisetty I will try this. And which index i should write in inputs.conf (Splunk UF integration) as Splunk will be monitoring the same file.

[thambisetty]

@payal23 ,

you can give any index based on your requirement, if you give different index other than indexA and indexB used in transforms, you will be ending up with total 3 indexes for same input. hope this clears your doubt.

up vote if my solution works for you.

[payal23]

Thanks for the explanation @thambisetty .. and its working!!! I added WRITE_META=true as well.

I have one more question: This configuration will work in both HF and Indexer, so which one should i use for this scenario?

Read this link and so I am confused: https://aditumpartners.com/5-splunk-myths-busted/

[thambisetty]

Data flow between universal forwarder and Indexer

universal forwarder -> Heavy forwarder(optional) -> Indexer

if you have heavy forwarder in place, I recommend using HF because the purpose of placing HF in between UF and Indexer is to parse/clean the data. Indexer is busy in serving requests coming from Search Head.

if you don't have HF in the flow , you can apply on Indexer.

Hope this helps...

up vote, if it solves your issue.