how to index locally and forward a specific sourcetype

[sylbaea]

Hello,

When events with a specific sourcetype arrive on my indexers, I would like to have both local indexing (default for any kind of sourcetype) but also forward them to another Splunk indexer.

So far I got this... It does properly forward this sourcetype to the external indexer.
But no longer index the events locally.

outputs.conf
[tcpout:externalIndexer]
server = external_indexer:9997

props.conf
[SourceTypeToForward]
TRANSFORMS-routing = sendToExternalIndexer

transforms.conf
[sendToExternalIndexer]
REGEX = .
DEST_KEY = _TCP_ROUTING
FORMAT = externalIndexer

How can I enhance this config to have both ?

[gcusello]

Hi sylbaea,
see at http://docs.splunk.com/Documentation/Splunk/latest/Forwarding/Routeandfilterdatad the part "Perform selective indexing and forwarding"
in details for locally index logs and forward a part of them you have to:
In outputs.conf, add the [indexAndForward] stanza:

[indexAndForward]
index=true
selectiveIndexing=true 

Note: This is a global stanza, and only needs to appear once in outputs.conf.
Include the target group stanzas for each set of receiving indexers:

[tcpout:<target_group>]
server = <ip address>:<port>, <ip address>:<port>, ...
... 

The forwarder uses the named in inputs.conf to route the inputs.

In inputs.conf, add the _INDEX_AND_FORWARD_ROUTING setting to the stanzas of each input that you want to index locally:

[input_stanza]
_INDEX_AND_FORWARD_ROUTING=<any_string>
...

Add the _TCP_ROUTING setting to the stanzas of each input that you want to forward:

[input_stanza]
_TCP_ROUTING=<target_group>
...

The is the name used in outputs.conf to specify the target group of receiving indexers.

Bye.
Giuseppe

[jbarlow_splunk · Accepted solution]

see :
https://docs.splunk.com/Documentation/Splunk/latest/Admin/Outputsconf#IndexAndForward_Processor-----

[sylbaea]

Thanks. I had a look already to this but it is unclear to me how it could apply to my case.

[jbarlow_splunk]

It has an example as well..

Perform selective indexing and forwarding

With a heavy forwarder only, you can index and store data locally, as well as

forward the data onwards to a receiving indexer. There are two ways to do

this:

1. In outputs.conf:

[tcpout]
defaultGroup = indexers

[indexAndForward]
index=true
selectiveIndexing=true

[tcpout:indexers]
server = 10.1.1.197:9997, 10.1.1.200:9997

2. In inputs.conf, Add _INDEX_AND_FORWARD_ROUTING for any data that you want

index locally, and

_TCP_ROUTING= for data to be forwarded.

[monitor:///var/log/messages/]
_INDEX_AND_FORWARD_ROUTING=local

[monitor:///var/log/httpd/]
_TCP_ROUTING=indexers

[sylbaea]

I got your point. Thanks 🙂
Will test