Routing and transforming to two different indexers?

[ldnail_at_TI]

Today I have a custom sourcetype = custom:access_combined this is routed in its entirety at the heavy forwarder to two different index clusters.

Indexer1 is the dev team, indexer2 is ops.

So the problem I'm running into is that I'd like to:
- route a full copy to indexer1
- for indexer2, run through transforms and drop a bunch of noise (like 75%) ops doesn't need to nullqueue

Any ideas on how to approach this? 

[ldnail_at_TI]

I didn't see that in the forest of trees... thanks.

[shivanshu1593 · Accepted solution]

Like this:

On your HF, go to props.conf or create one in the directory $SPLUNK_HOME/etc/system/local or $SPLUNK_HOME/etc/apps/<app_name>/local and put the following:

[custom:access_combined]
TRANSFORMS-routing=devRouting
TRANSFORMS-routing1=opsRouting

 In the same path, edit or create transforms.conf and put the following:

[devRouting]
REGEX= .
DEST_KEY=_TCP_ROUTING
FORMAT=devGroup

[opsRouting]
REGEX= <put the regex to select & identify the data that you want to send to the ops indexer>
DEST_KEY=_TCP_ROUTING
FORMAT=opsGroup

Then in outputs.conf, append the following changes:

[tcpout:devGroup]
server=<ip of dev indexer>:<port number>

[tcpout:opsGroup]
server=<ip of ops indexer>:<port number>

Restart splunkd and check the results. If the changes aren't reflecting, please check the following:

1. Your regex is correct.

2. Entries about the IP and port are correct 

3. You or the team has the rights to access data.

4. Run btool and see what configurations are loaded for the app and sourcetype, for which we made the changes.

Hope this helps. Let me know if it works.

Thanks,

S

****If it helped, please upvote and accept it as a solution. It helps other Splunkers to find the solutions easily****

[iguardia]

in the same situation, how can I change de index for each remote indexer?

For example,  ops_index for ops and dev_index for dev, as follow:

props.conf:

[custom:access_combined]
TRANSFORMS-routing=devRouting,devIndex
TRANSFORMS-routing1=opsRouting,opsIndex

transforms:

[devRouting]
REGEX= .
DEST_KEY=_TCP_ROUTING
FORMAT=devGroup

[opsRouting]
REGEX= <put the regex to select & identify the data that you want to send to the ops indexer>
DEST_KEY=_TCP_ROUTING
FORMAT=opsGroup

[devIndex]
REGEX=.
DEST_KEY=_MetaData:Index
FORMAT=dev_index

[opsIndex]
REGEX=.
DEST_KEY=_MetaData:Index
FORMAT=ops_index

is right?