Getting Data In

Index internal logs locally and forward all other logs

k31453
Explorer

As title suggest, i want to index internal logs only and forwards all other logs to forwarders or idxs.

Here is the setup :

  • I have one cluster and three indexes setup seperately outside cluster.
  • Cluster has CM, SH and three indexers.
  • Those Three indexers i want to use as Heavy forwarder to send all logs out to external indexes

Following is default output.conf:

[tcpout]
maxQueueSize = auto
forwardedindex.0.whitelist = .*
forwardedindex.1.blacklist = _.*
forwardedindex.2.whitelist = (_audit|_internal|_introspection|_telemetry|_metrics|_metrics_rollup)
forwardedindex.filter.disable = false
indexAndForward = false

Here is what I have done outputs.conf

 

[tcpout]
defaultGroup=noforward
disabled=false

[indexAndForward]
index=true
selectiveIndexing=true

[tcpout:forwarders]
server:<forwarders>:9997

 

 

 
Below is my props.conf

 

 

[default]
TRANSFORMS-forwardit = forwardit

[host::*.foo.splunk.com]
TRANSFORMS-routing = indexing

 

 


Below is transforms.conf

 

 

[forwardit]
REGEX = .
DEST_KEY = _TCP_ROUTING
FORMAT = forwarders

[indexing]
REGEX = .
DEST_KEY = _INDEX_AND_FORWARD_ROUTING
FORMAT = local

 

 

 
Essentially all internal indexes should stay within cluster indexes but rest of index or logs forwarded to external indexes.

Labels (2)
0 Karma

thambisetty
SplunkTrust
SplunkTrust

@k31453 

I believe you are looking for below: Note: you can only index _internal logs using this method.

https://docs.splunk.com/Documentation/Splunk/8.1.0/Forwarding/Routeandfilterdatad#Selective_indexing...

 

————————————
If this helps, give a like below.
0 Karma

k31453
Explorer

Well. This tells me i have to use inputs.conf to ensure routing. By default I want to forward logs. But if i see internal logs i will index it and not forward it. This basically is telling me i have to put _INDEX_AND_FORWARD_ROUTING on all internal inputs.conf this can cause the issue. 

0 Karma

k31453
Explorer

For me by default i want to forward new indexes created and internal indexes has to be indexed locally. My thoughts is , setup tcpgroup for forwarders and in outputs.conf and inputs.conf i should modify but not sure how.

0 Karma

k31453
Explorer

Hi, if the intention is to index all internal indexes, i have set _INDEX_AND_FORWARD_ROUTING and 

_TCP_ROUTING which can cause the issue.

0 Karma

thambisetty
SplunkTrust
SplunkTrust

@k31453 

The Splunk Doc is very much detailed on the question you have asked. check it out using below link.

https://docs.splunk.com/Documentation/Splunk/8.1.0/Forwarding/Routeandfilterdatad#Perform_selective_...

————————————
If this helps, give a like below.
0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud | Unified Identity - Now Available for Existing Splunk ...

Raise your hand if you’ve already forgotten your username or password when logging into an account. (We can’t ...

Index This | How many sides does a circle have?

February 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

Registration for Splunk University is Now Open!

Are you ready for an adventure in learning?   Brace yourselves because Splunk University is back, and it's ...