Knowledge Management

Routing summary indexes to specific indexer based on summary index name

melonman
Motivator

Hi,

Could anyone help me with configuration for the following?

  • summary indexes created on search head layer to indexer layer (outputs.conf)
  • send summary1 to indexer1 and 2 (clone)
  • send summary2 to indexer3 and 4 (clone)
  • send _internal index to indexer1,2,3,4 (spray)

Actual configuration will be more complicated, but I would like to know how to do this as an example of summary index routing.

Any comment would be really appreciated.

0 Karma

melonman
Motivator

The following setting worked, but I am still not sure about blockOnCloning and some other important parameters for cloning in outputs.conf can work for _TCP_ROUTING in transforms.conf...

Anyway, this is what I've got sp far.

outputs.conf

[tcpout]
defaultGroup = sprayAll

[tcpout:sprayAll]
server = 127.0.0.1:19997,127.0.0.1:29997,127.0.0.1:39997,127.0.0.1:49997
autoLB = true
autoLBFrequency = 13


[tcpout:idx1_9997]
server = 127.0.0.1:19997

[tcpout:idx2_9997]
server = 127.0.0.1:29997

[tcpout:idx3_9997]
server = 127.0.0.1:39997

[tcpout:idx4_9997]
server = 127.0.0.1:49997

props.conf

[stash_new]
TRANSFORMS-routing = summary1,summary2

transforms.conf

[summary1]
SOURCE_KEY = _MetaData:Index
REGEX = summary1
DEST_KEY = _TCP_ROUTING
FORMAT = idx1_9997,idx2_9997

[summary2]
SOURCE_KEY = _MetaData:Index
REGEX = summary2
DEST_KEY = _TCP_ROUTING
FORMAT = idx3_9997,idx4_9997

guitarmansevevn
Engager

This did the trick for us. Note that all the other summary indexing will use the default routing. This is exactly what we needed to happen.

0 Karma

somesoni2
Revered Legend

See this link for details on to selectively forward data from an index.

http://docs.splunk.com/Documentation/Splunk/latest/Forwarding/Routeandfilterdatad#Filter_data_by_tar...

I believe something like this should work
outputs.conf on Search Head

[tcpout:indexer1]
server=server1:9997
#Disable the current filters from the defaults outputs.conf
forwardedindex.0.whitelist = 
forwardedindex.1.blacklist =
forwardedindex.2.whitelist =
forwardedindex.0.whitelist = summary1
forwardedindex.1.whitelist = _internal

[tcpout:indexer2]
server=server1:9997
#Disable the current filters from the defaults outputs.conf
forwardedindex.0.whitelist = 
forwardedindex.1.blacklist =
forwardedindex.2.whitelist =
forwardedindex.0.whitelist = summary1
forwardedindex.1.whitelist = _internal

[tcpout:indexer3]
server=server1:9997
#Disable the current filters from the defaults outputs.conf
forwardedindex.0.whitelist = 
forwardedindex.1.blacklist =
forwardedindex.2.whitelist =
forwardedindex.0.whitelist = summary2
forwardedindex.1.whitelist = _internal

[tcpout:indexer4]
server=server1:9997
#Disable the current filters from the defaults outputs.conf
forwardedindex.0.whitelist = 
forwardedindex.1.blacklist =
forwardedindex.2.whitelist =
forwardedindex.0.whitelist = summary2
forwardedindex.1.whitelist = _internal
0 Karma

melonman
Motivator

I thought the same thing, but actually the filter is only applicable to [tcpout] stanza, as stated in outputs.conf.spec.

#----Index Filter Settings.
# These attributes are only applicable under the global [tcpout] stanza.
# This filter does not work if it is created under any other stanza.
forwardedindex.<n>.whitelist = <regex>
forwardedindex.<n>.blacklist = <regex>

Probably what Splunk can do with this configuration is to select which index to be fotwarded or not. and this is not for selecting a destinating indexers... maybe.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

 (view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...