Heavy Forwarder routing to two seperate indexer clusters

[fisuser1]

I have a HF forwarding specific sourcetypes to two different indexer clusters. However, it does not seem to be working even though I have followed routing and filter data approach. All data seems to be forwarding to all indexers. (seeing errors on the second set of indexers since the index does not exist, which is expected) I've also confirmed there isn't any conflicting inputs taking precedence in system/local.

https://docs.splunk.com/Documentation/Splunk/7.3.1/Forwarding/Routeandfilterdatad

Any suggestions?

outputs
[tcpout]
indexAndForward = 0

[tcpout]
defaultGroup = fis_search_peers,profile_search_peers

[tcpout:fis_search_peers]
server = idx1:9997, idx2:9997, idx3:9997, idx4:9997, idx5:9997, idx6:9997, idx7:9997, idx8:9997
autoLBFrequency=15
autoLB=true
useACK=true
forceTimebasedAutoLB = true

[tcpout:profile_search_peers]
server = f_idx1:9997, f_idx2:9997, f_idx3:9997
autoLBFrequency=15
autoLB=true
useACK=true
forceTimebasedAutoLB = true

inputs
[integrated_payables]
connection = prd_integrated_payables
disabled = 0
index = integrated_payables
index_time_mode = dbColumn
input_timestamp_column_number = 2
interval = 900
mode = rising
query = SELECT [Id] ,\
[Date] ,\
[Thread] ,\
[Level] ,\
Logger ,\
NestedContext ,\
[Message] ,\
[Exception] ,\
[HostName] ,\
[Thread_Context_ID]\
FROM "Logging"."dbo"."Log"\
WHERE Date > ?\
ORDER BY Date ASC
sourcetype = prd_int_pay_payspan_logging_db
tail_rising_column_number = 2
_TCP_ROUTING = fis_search_peers

[monitor:///ma/xpressng/logs/WAS85//MA_XpressNG_Prd_ma.log]
sourcetype = prd_mba_infinity_application
index = mba_infinity
_TCP_ROUTING = profile_search_peers

[renjith_nair · Accepted solution]

@fisuser1 ,

You have set defaultGroup to both indexer groups and as per documentation, it sends events to all specified target group

https://docs.splunk.com/Documentation/Forwarder/7.3.1/Forwarder/Configureforwardingwithoutputs.conf#...

Set default target groups in outputs.conf

The defaultGroup attribute lets you set default groups for automatic forwarding at the global level, in your [tcpout] stanza.

The defaultGroup specifies one or more target groups that you define later in tcpout:<target_group> stanzas. The forwarder sends all events to the specified groups.    
[tcpout]
defaultGroup= <target_group1>, <target_group2>, ...
If you do not want to forward data automatically, do not set the defaultGroup attribute.

[fisuser1]

somehow I completely overlooked this, thank you @renjith.nair, I believe this will do exactly what I am looking for in breaking up this traffic. Thank you!