I want to send my log files to two different Splunk instances, but the data is only only being sent to one of the two environments.
[tcpout] defaultGroup=noredirect disabled = false maxQueueSize = 500KB [tcpout:twd104] server = 10.9.3.35:9997 [tcpout:splunk_prod] server = splunk-indexer01.rotterdam.local:9997,splunk-indexer02.rotterdam.local:9997
[monitor:///var/log/mule/.../*] _TCP_ROUTING=splunk_prod disabled = false followTail = 0 sourcetype = log4j index = acc_abkr [monitor:///var/log/mule/.../*] _TCP_ROUTING=twd104 disabled = false followTail = 0 sourcetype = log4j
The data is only send to splunk_prod.
Why is this not working?
I agree with hgrow. I don't think you can have 2 stanzas dealing with the same log source.
So instead of using _TCP_ROUTING in inputs, use props and transforms.
props.conf (Not sure about the source.)
[source::///var/log/mule/.../*] TRANSFORMS-routingto2splunks = log_to_splunk1, log_to_splunk2
[log_to_splunk1] REGEX = . DEST_KEY = _TCP_ROUTING FORMAT = splunk_prod [log_to_splunk2] REGEX = . DEST_KEY = _SYSLOG_ROUTING FORMAT = twd104
My guess is that Splunk ignores your second input because all data is already read with the first input since both inputs are matching the same monitoring pattern.
You don't want two Inputs for the same source. What might help you is a second target group for your input.
What you need is something like:
[tcpout] defaultGroup= <target_group1> , <target_group2> [tcpout:<target_group1>] server= <receiving_server1> , <receiving_server2> [tcpout:<target_group2>] server= <receiving_server3>
I'm not sure if your problem is stil relevant, but i've done some tests. Here is a working configuration:
[monitor:///opt/splunkdata/testdata.log] disabled = false index = main sourcetype = test _TCP_ROUTING = indexer1,indexer2
[tcpout] indexAndForward=true defaultGroup=discard [tcpout:indexer1] server=192.168.111.246:9997 [tcpout:indexer2] server=192.168.111.245:9997 [tcpout:discard] disabled = true
In addition to my previous posts you define multiple tcpout_groups for the _TCP_ROUTING directly in your input. In this way you can use default routing functionality within the default stanza tcpout for something else.
That's strange. I wasnt able to rebuild your problem but according to the docs its what you need for data cloning.
To perform data cloning, specify multiple target groups, each in its own stanza. In data cloning, the forwarder sends copies of all its events to the receivers in two or more target groups. Data cloning usually results in similar, but not necessarily exact, copies of data on the receiving indexers. Here's an example of how you set up data cloning:
[tcpout] defaultGroup=indexer1,indexer2 [tcpout:indexer1] server=10.1.1.197:9997 [tcpout:indexer2] server=10.1.1.200:9997
The forwarder will send duplicate data streams to the servers specified in both the indexer1 and indexer2 target groups.
Can you post your changed inputs.conf and outputs.conf ?