Getting Data In

Index internal logs locally and forward all other logs

k31453
Explorer

As title suggest, i want to index internal logs only and forwards all other logs to forwarders or idxs.

Here is the setup :

  • I have one cluster and three indexes setup seperately outside cluster.
  • Cluster has CM, SH and three indexers.
  • Those Three indexers i want to use as Heavy forwarder to send all logs out to external indexes

Following is default output.conf:

[tcpout]
maxQueueSize = auto
forwardedindex.0.whitelist = .*
forwardedindex.1.blacklist = _.*
forwardedindex.2.whitelist = (_audit|_internal|_introspection|_telemetry|_metrics|_metrics_rollup)
forwardedindex.filter.disable = false
indexAndForward = false

Here is what I have done outputs.conf

 

[tcpout]
defaultGroup=noforward
disabled=false

[indexAndForward]
index=true
selectiveIndexing=true

[tcpout:forwarders]
server:<forwarders>:9997

 

 

 
Below is my props.conf

 

 

[default]
TRANSFORMS-forwardit = forwardit

[host::*.foo.splunk.com]
TRANSFORMS-routing = indexing

 

 


Below is transforms.conf

 

 

[forwardit]
REGEX = .
DEST_KEY = _TCP_ROUTING
FORMAT = forwarders

[indexing]
REGEX = .
DEST_KEY = _INDEX_AND_FORWARD_ROUTING
FORMAT = local

 

 

 
Essentially all internal indexes should stay within cluster indexes but rest of index or logs forwarded to external indexes.

Labels (2)
0 Karma

thambisetty
Super Champion

@k31453 

I believe you are looking for below: Note: you can only index _internal logs using this method.

https://docs.splunk.com/Documentation/Splunk/8.1.0/Forwarding/Routeandfilterdatad#Selective_indexing...

 

————————————
If this helps, give a like below.
0 Karma

k31453
Explorer

Well. This tells me i have to use inputs.conf to ensure routing. By default I want to forward logs. But if i see internal logs i will index it and not forward it. This basically is telling me i have to put _INDEX_AND_FORWARD_ROUTING on all internal inputs.conf this can cause the issue. 

0 Karma

k31453
Explorer

For me by default i want to forward new indexes created and internal indexes has to be indexed locally. My thoughts is , setup tcpgroup for forwarders and in outputs.conf and inputs.conf i should modify but not sure how.

0 Karma

k31453
Explorer

Hi, if the intention is to index all internal indexes, i have set _INDEX_AND_FORWARD_ROUTING and 

_TCP_ROUTING which can cause the issue.

0 Karma

thambisetty
Super Champion

@k31453 

The Splunk Doc is very much detailed on the question you have asked. check it out using below link.

https://docs.splunk.com/Documentation/Splunk/8.1.0/Forwarding/Routeandfilterdatad#Perform_selective_...

————————————
If this helps, give a like below.
0 Karma