Getting Data In
Highlighted

How to use sourcetype to route data from a heavy forwarder to different indexer cluster groups ?

Engager

Hi there,

We're trying to have a splunk forwarder to send data to an intermediate splunk heavy forwarder that
clones data to different indexer cluster groups based on the sourcetype.

sourcetype1 send data to indexercluster1 (default routing)
sourcetype2 send data to indexercluster1 and indexercluster2

We have set the following below, but data is only indexed in indexercluster1 for some reason.
Never sent to indexercluster2 for sourcetype2 as wanted.

Any idea why ?

Thanks a lot for any help

props.conf

[sourcetype2]
TRANSFORMS-routing=TR_routing_sourcetype2 

transforms.conf

[TR_routing_sourcetype2]
REGEX=.
DEST_KEY=_TCP_ROUTING
FORMAT=indexercluster1,indexercluster2

outputs.conf

[tcpout]
forwardedindex.0.whitelist = .*
forwardedindex.1.blacklist = 
forwardedindex.2.whitelist = 
forwardedindex.filter.disable = false

defaultGroup = indexercluster1

indexAndForward = false

[tcpout:indexercluster1]
disabled = false
maxQueueSize = 6MB
server = 10.0.1.1:9997,10.0.1.2:9997,10.0.1.3:9997
sslCertPath = $SPLUNK_HOME/etc/auth/server.pem
sslPassword = somepassword
sslRootCAPath = $SPLUNK_HOME/etc/auth/cacert.pem
sslVerifyServerCert = false
useACK = true

[tcpout:indexercluster2]
disabled = false
maxQueueSize = 6MB
server = 10.0.2.1:9997,10.0.2.2:9997,10.0.2.3:9997
sslCertPath = $SPLUNK_HOME/etc/auth/server.pem
sslPassword = somepassword
sslRootCAPath = $SPLUNK_HOME/etc/auth/cacert.pem
sslVerifyServerCert = false
useACK = true

inputs.conf

[default]
#_TCP_ROUTING = *

[SSL]
rootCA = $SPLUNK_HOME/etc/auth/cacert.pem
serverCert = $SPLUNK_HOME/etc/auth/server.pem
password = somepassword

[splunktcp-ssl:9997]
0 Karma
Highlighted

Re: How to use sourcetype to route data from a heavy forwarder to different indexer cluster groups ?

Splunk Employee
Splunk Employee

Have you tried breaking it into two individual transforms? e.g.

props.conf

[sourcetype2]
TRANSFORMS-routing=TRroutingsourcetype2IDXC1,TRroutingsourcetype2IDXC2

transforms.conf:

[TRroutingsourcetype2IDXC1]
REGEX=.
DESTKEY=TCP_ROUTING
FORMAT=indexercluster1

[TRroutingsourcetype2IDXC2]
REGEX=.
DESTKEY=TCP_ROUTING
FORMAT=indexercluster2

Highlighted

Re: How to use sourcetype to route data from a heavy forwarder to different indexer cluster groups ?

Engager

Thanks dgrubb, much appreciated. We tried this, and we still have data forwarded only to the defaultGroup indexercluster1. No sourcetype2 in indexercluster2.

It was working before when inputs.conf had the TCPROUTING = *, but then all data was sent to all outputs.conf groups whatever the sourcetype, which was not the behaviour we wanted either.

We would like to send all data by default to indexercluster1, and for a selected number of sourcetypes to go to indexercluster1 and indexercluster2.

Any help would be appreciated.

0 Karma
Highlighted

Re: How to use sourcetype to route data from a heavy forwarder to different indexer cluster groups ?

Splunk Employee
Splunk Employee

Hi,

which version are you using?

In 6.4.3, according to inputs.conf you can use multiple groups... haven't tested.

Don't know whether this works in props/transforms.

One other option I have in mind is to use CLONE_SOURCETYPE and route this new sourcetype to an additional group.

It's possible to do this

props.conf:

[orig_sourcetype]
TRANSFORMS-clone = clone_orig

[cloned_sourcetype]
TRANSFORMS-route = route_clone

transforms.conf

[clone_orig]
REGEX = (.*)
FORMAT = $1
DEST_KEY = _raw
CLONE_SOURCETYPE = cloned_sourcetype

[route_clone]
REGEX = .
DEST_KEY=_TCP_ROUTING
FORMAT=indexercluster2

outputs.conf should stay the same...? Does it work for you?
You should be able to add more transforms for the "new" sourcetype... maybe even renaming it again?
Or use a sourcetype alias on the second indexers group.

HTH,

Holger

0 Karma