We're trying to have a splunk forwarder to send data to an intermediate splunk heavy forwarder that
clones data to different indexer cluster groups based on the sourcetype.
sourcetype1 send data to indexercluster1 (default routing)
sourcetype2 send data to indexercluster1 and indexercluster2
We have set the following below, but data is only indexed in indexercluster1 for some reason.
Never sent to indexercluster2 for sourcetype2 as wanted.
Any idea why ?
Thanks a lot for any help
[TR_routing_sourcetype2] REGEX=. DEST_KEY=_TCP_ROUTING FORMAT=indexercluster1,indexercluster2
[tcpout] forwardedindex.0.whitelist = .* forwardedindex.1.blacklist = forwardedindex.2.whitelist = forwardedindex.filter.disable = false defaultGroup = indexercluster1 indexAndForward = false [tcpout:indexercluster1] disabled = false maxQueueSize = 6MB server = 10.0.1.1:9997,10.0.1.2:9997,10.0.1.3:9997 sslCertPath = $SPLUNK_HOME/etc/auth/server.pem sslPassword = somepassword sslRootCAPath = $SPLUNK_HOME/etc/auth/cacert.pem sslVerifyServerCert = false useACK = true [tcpout:indexercluster2] disabled = false maxQueueSize = 6MB server = 10.0.2.1:9997,10.0.2.2:9997,10.0.2.3:9997 sslCertPath = $SPLUNK_HOME/etc/auth/server.pem sslPassword = somepassword sslRootCAPath = $SPLUNK_HOME/etc/auth/cacert.pem sslVerifyServerCert = false useACK = true
[default] #_TCP_ROUTING = * [SSL] rootCA = $SPLUNK_HOME/etc/auth/cacert.pem serverCert = $SPLUNK_HOME/etc/auth/server.pem password = somepassword [splunktcp-ssl:9997]
which version are you using?
In 6.4.3, according to inputs.conf you can use multiple groups... haven't tested.
Don't know whether this works in props/transforms.
One other option I have in mind is to use CLONE_SOURCETYPE and route this new sourcetype to an additional group.
It's possible to do this
[orig_sourcetype] TRANSFORMS-clone = clone_orig [cloned_sourcetype] TRANSFORMS-route = route_clone
[clone_orig] REGEX = (.*) FORMAT = $1 DEST_KEY = _raw CLONE_SOURCETYPE = cloned_sourcetype [route_clone] REGEX = . DEST_KEY=_TCP_ROUTING FORMAT=indexercluster2
outputs.conf should stay the same...? Does it work for you?
You should be able to add more transforms for the "new" sourcetype... maybe even renaming it again?
Or use a sourcetype alias on the second indexers group.
Have you tried breaking it into two individual transforms? e.g.
Thanks dgrubb, much appreciated. We tried this, and we still have data forwarded only to the defaultGroup indexercluster1. No sourcetype2 in indexercluster2.
It was working before when inputs.conf had the TCPROUTING = *, but then all data was sent to all outputs.conf groups whatever the sourcetype, which was not the behaviour we wanted either.
We would like to send all data by default to indexercluster1, and for a selected number of sourcetypes to go to indexercluster1 and indexercluster2.
Any help would be appreciated.