How can we send data to 2 different groups of indexers?

[splunker9999]

Hi

We are looking to forward same data to different indexers and we did the below steps for this.

We have 2 apps for outputs and each has one outputs.conf

1) output_ app1

     outputs.conf   
     [tcpout:indexers_prod]
    server=server.corp:9197

2) output_app2

     AWS_outputs.conf
     [tcpout:AWS_indexers_prod]
    server=server.corp:9197

We now created a new server class (data_inputs).
Created -> Serverclass -> data_inputs

Added above apps (app1 &app2 to server class), and also added a 3rd app (inputs app)

3) Created 3rd app (inputs app) and added below lines to inputs.conf

inputs.conf:

    [monitor:///logs/svc_cml_*/jobs/SCDB2/.../*.log]
    _TCP_ROUTING=indexers_prod;AWS_indexers_prod
    disabled=false
    index=sc_preprod

4) added Clients to serverclass
5) deployed these apps to clients.

We did above steps, but we are seeing data only on one of the groups of indexers.

Can someone help if there is any thing we missed here?

Thanks.

[lguinn2 · Accepted solution]

First, the only name for outputs.conf is outputs.conf. AWS_outputs.conf will not be read or recognized. So I hope that you just gave that name so that we could distinguish the two files in this question. If not, you will need to make both names outputs.conf

Second, if you want, you can combine both stanzas into the same outputs.conf:

[tcpout]
defaultGroup = indexers_prod

[tcpout:indexers_prod]
server=server1.corp:9197

[tcpout:AWS_indexers_prod]
server=server2.corp:9197

Notice that I set indexers_prod as the default output stanza. Any input that does not specify a routing will go to indexers_prod
Also, I made the different server entries correspond to different servers - otherwise, I don't get the point. But you can still have two separate outputs.conf files if you prefer.

Third, whether you combine the outputs.conf or not, your inputs.conf should look like this (commas, not semicolons)

[monitor:///logs/svc_cml_*/jobs/SCDB2/.../*.log]
_TCP_ROUTING=indexers_prod,AWS_indexers_prod

I think this will work.

[splunker9999]

Thank you,

Changed semicolon to comma and logs ingested to both places like a champ 🙂

[monitor:///logs/svc_cml_*/jobs/SCDB2/.../*.log]
 _TCP_ROUTING=indexers_prod,AWS_indexers_prod

[iamkilarunaresh]

How can we put the index names here?

[bzam]

Wouldn't you just specify the index name like this:

[monitor:///logs/svc_cml_*/jobs/SCDB2/.../*.log]
 _TCP_ROUTING=indexers_prod,AWS_indexers_prod
index=foo