Knowledge Management

Routing summary indexes to specific indexer based on summary index name

Motivator

Hi,

Could anyone help me with configuration for the following?

  • summary indexes created on search head layer to indexer layer (outputs.conf)
  • send summary1 to indexer1 and 2 (clone)
  • send summary2 to indexer3 and 4 (clone)
  • send _internal index to indexer1,2,3,4 (spray)

Actual configuration will be more complicated, but I would like to know how to do this as an example of summary index routing.

Any comment would be really appreciated.

0 Karma

Motivator

The following setting worked, but I am still not sure about blockOnCloning and some other important parameters for cloning in outputs.conf can work for _TCP_ROUTING in transforms.conf...

Anyway, this is what I've got sp far.

outputs.conf

[tcpout]
defaultGroup = sprayAll

[tcpout:sprayAll]
server = 127.0.0.1:19997,127.0.0.1:29997,127.0.0.1:39997,127.0.0.1:49997
autoLB = true
autoLBFrequency = 13


[tcpout:idx1_9997]
server = 127.0.0.1:19997

[tcpout:idx2_9997]
server = 127.0.0.1:29997

[tcpout:idx3_9997]
server = 127.0.0.1:39997

[tcpout:idx4_9997]
server = 127.0.0.1:49997

props.conf

[stash_new]
TRANSFORMS-routing = summary1,summary2

transforms.conf

[summary1]
SOURCE_KEY = _MetaData:Index
REGEX = summary1
DEST_KEY = _TCP_ROUTING
FORMAT = idx1_9997,idx2_9997

[summary2]
SOURCE_KEY = _MetaData:Index
REGEX = summary2
DEST_KEY = _TCP_ROUTING
FORMAT = idx3_9997,idx4_9997

This did the trick for us. Note that all the other summary indexing will use the default routing. This is exactly what we needed to happen.

0 Karma

Revered Legend

See this link for details on to selectively forward data from an index.

http://docs.splunk.com/Documentation/Splunk/latest/Forwarding/Routeandfilterdatad#Filter_data_by_tar...

I believe something like this should work
outputs.conf on Search Head

[tcpout:indexer1]
server=server1:9997
#Disable the current filters from the defaults outputs.conf
forwardedindex.0.whitelist = 
forwardedindex.1.blacklist =
forwardedindex.2.whitelist =
forwardedindex.0.whitelist = summary1
forwardedindex.1.whitelist = _internal

[tcpout:indexer2]
server=server1:9997
#Disable the current filters from the defaults outputs.conf
forwardedindex.0.whitelist = 
forwardedindex.1.blacklist =
forwardedindex.2.whitelist =
forwardedindex.0.whitelist = summary1
forwardedindex.1.whitelist = _internal

[tcpout:indexer3]
server=server1:9997
#Disable the current filters from the defaults outputs.conf
forwardedindex.0.whitelist = 
forwardedindex.1.blacklist =
forwardedindex.2.whitelist =
forwardedindex.0.whitelist = summary2
forwardedindex.1.whitelist = _internal

[tcpout:indexer4]
server=server1:9997
#Disable the current filters from the defaults outputs.conf
forwardedindex.0.whitelist = 
forwardedindex.1.blacklist =
forwardedindex.2.whitelist =
forwardedindex.0.whitelist = summary2
forwardedindex.1.whitelist = _internal
0 Karma

Motivator

I thought the same thing, but actually the filter is only applicable to [tcpout] stanza, as stated in outputs.conf.spec.

#----Index Filter Settings.
# These attributes are only applicable under the global [tcpout] stanza.
# This filter does not work if it is created under any other stanza.
forwardedindex.<n>.whitelist = <regex>
forwardedindex.<n>.blacklist = <regex>

Probably what Splunk can do with this configuration is to select which index to be fotwarded or not. and this is not for selecting a destinating indexers... maybe.

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!