We are looking to forward same data to different indexers and we did the below steps for this.
We have 2 apps for outputs and each has one outputs.conf
1) output_ app1
outputs.conf [tcpout:indexers_prod] server=server.corp:9197
AWS_outputs.conf [tcpout:AWS_indexers_prod] server=server.corp:9197
We now created a new server class (datainputs).
Created -> Serverclass -> datainputs
Added above apps (app1 &app2 to server class), and also added a 3rd app (inputs app)
3) Created 3rd app (inputs app) and added below lines to inputs.conf
[monitor:///logs/svc_cml_*/jobs/SCDB2/.../*.log] _TCP_ROUTING=indexers_prod;AWS_indexers_prod disabled=false index=sc_preprod
4) added Clients to serverclass
5) deployed these apps to clients.
We did above steps, but we are seeing data only on one of the groups of indexers.
Can someone help if there is any thing we missed here?
First, the only name for outputs.conf is outputs.conf. AWS_outputs.conf will not be read or recognized. So I hope that you just gave that name so that we could distinguish the two files in this question. If not, you will need to make both names outputs.conf
Second, if you want, you can combine both stanzas into the same outputs.conf:
[tcpout] defaultGroup = indexers_prod [tcpout:indexers_prod] server=server1.corp:9197 [tcpout:AWS_indexers_prod] server=server2.corp:9197
Notice that I set
indexers_prod as the default output stanza. Any input that does not specify a routing will go to
Also, I made the different server entries correspond to different servers - otherwise, I don't get the point. But you can still have two separate outputs.conf files if you prefer.
Third, whether you combine the outputs.conf or not, your inputs.conf should look like this (commas, not semicolons)
I think this will work.
Changed semicolon to comma and logs ingested to both places like a champ 🙂
Wouldn't you just specify the index name like this:
[monitor:///logs/svc_cml_*/jobs/SCDB2/.../*.log] _TCP_ROUTING=indexers_prod,AWS_indexers_prod index=foo