Index internal logs locally and forward all other logs

[k31453]

As title suggest, i want to index internal logs only and forwards all other logs to forwarders or idxs.

Here is the setup :

• I have one cluster and three indexes setup seperately outside cluster.
• Cluster has CM, SH and three indexers.
• Those Three indexers i want to use as Heavy forwarder to send all logs out to external indexes
  
  

Following is default output.conf:

[tcpout]
maxQueueSize = auto
forwardedindex.0.whitelist = .*
forwardedindex.1.blacklist = _.*
forwardedindex.2.whitelist = (_audit|_internal|_introspection|_telemetry|_metrics|_metrics_rollup)
forwardedindex.filter.disable = false
indexAndForward = false

Here is what I have done outputs.conf

[tcpout]
defaultGroup=noforward
disabled=false

[indexAndForward]
index=true
selectiveIndexing=true

[tcpout:forwarders]
server:<forwarders>:9997

 
Below is my props.conf

[default]
TRANSFORMS-forwardit = forwardit

[host::*.foo.splunk.com]
TRANSFORMS-routing = indexing

Below is transforms.conf

[forwardit]
REGEX = .
DEST_KEY = _TCP_ROUTING
FORMAT = forwarders

[indexing]
REGEX = .
DEST_KEY = _INDEX_AND_FORWARD_ROUTING
FORMAT = local

 
Essentially all internal indexes should stay within cluster indexes but rest of index or logs forwarded to external indexes.

[thambisetty]

@k31453 

I believe you are looking for below: Note: you can only index _internal logs using this method.

https://docs.splunk.com/Documentation/Splunk/8.1.0/Forwarding/Routeandfilterdatad#Selective_indexing...

[k31453]

Well. This tells me i have to use inputs.conf to ensure routing. By default I want to forward logs. But if i see internal logs i will index it and not forward it. This basically is telling me i have to put _INDEX_AND_FORWARD_ROUTING on all internal inputs.conf this can cause the issue. 

[k31453]

For me by default i want to forward new indexes created and internal indexes has to be indexed locally. My thoughts is , setup tcpgroup for forwarders and in outputs.conf and inputs.conf i should modify but not sure how.

[k31453]

Hi, if the intention is to index all internal indexes, i have set _INDEX_AND_FORWARD_ROUTING and 

_TCP_ROUTING which can cause the issue.

[thambisetty]

@k31453 

The Splunk Doc is very much detailed on the question you have asked. check it out using below link.

https://docs.splunk.com/Documentation/Splunk/8.1.0/Forwarding/Routeandfilterdatad#Perform_selective_...