topic Re: routing and transforming to two different indexers in Splunk Search

Routing and transforming to two different indexers?

ldnail_at_TI — Thu, 20 Jul 2023 17:36:47 GMT

Today I have a custom sourcetype = custom:access_combined this is routed in its entirety at the heavy forwarder to two different index clusters.

Indexer1 is the dev team, indexer2 is ops.

So the problem I'm running into is that I'd like to:
- route a full copy to indexer1
- for indexer2, run through transforms and drop a bunch of noise (like 75%) ops doesn't need to nullqueue

Any ideas on how to approach this?

Re: routing and transforming to two different indexers

shivanshu1593 — Wed, 18 Aug 2021 19:53:36 GMT

Like this:

On your HF, go to props.conf or create one in the directory $SPLUNK_HOME/etc/system/local or $SPLUNK_HOME/etc/apps/<app_name>/local and put the following:

[custom:access_combined]
TRANSFORMS-routing=devRouting
TRANSFORMS-routing1=opsRouting

In the same path, edit or create transforms.conf and put the following:

[devRouting]
REGEX= .
DEST_KEY=_TCP_ROUTING
FORMAT=devGroup

[opsRouting]
REGEX= <put the regex to select & identify the data that you want to send to the ops indexer>
DEST_KEY=_TCP_ROUTING
FORMAT=opsGroup

Then in outputs.conf, append the following changes:

[tcpout:devGroup]
server=<ip of dev indexer>:<port number>

[tcpout:opsGroup]
server=<ip of ops indexer>:<port number>

Restart splunkd and check the results. If the changes aren't reflecting, please check the following:

1. Your regex is correct.

2. Entries about the IP and port are correct

3. You or the team has the rights to access data.

4. Run btool and see what configurations are loaded for the app and sourcetype, for which we made the changes.

Hope this helps. Let me know if it works.

Thanks,

****If it helped, please upvote and accept it as a solution. It helps other Splunkers to find the solutions easily****

Re: routing and transforming to two different indexers

ldnail_at_TI — Wed, 18 Aug 2021 21:24:18 GMT

I didn't see that in the forest of trees... thanks.

Re: routing and transforming to two different indexers

iguardia — Thu, 20 Jul 2023 15:44:51 GMT

in the same situation, how can I change de index for each remote indexer?

For example, ops_index for ops and dev_index for dev, as follow:

props.conf:

[custom:access_combined] TRANSFORMS-routing=devRouting,devIndex TRANSFORMS-routing1=opsRouting,opsIndex

transforms:

[devRouting] REGEX= . DEST_KEY=_TCP_ROUTING FORMAT=devGroup [opsRouting] REGEX= <put the regex to select & identify the data that you want to send to the ops indexer> DEST_KEY=_TCP_ROUTING FORMAT=opsGroup [devIndex] REGEX=. DEST_KEY=_MetaData:Index FORMAT=dev_index [opsIndex] REGEX=. DEST_KEY=_MetaData:Index FORMAT=ops_index

is right?