As title suggest, i want to index internal logs only and forwards all other logs to forwarders or idxs.
Here is the setup :
Following is default output.conf:
[tcpout] maxQueueSize = auto forwardedindex.0.whitelist = .* forwardedindex.1.blacklist = _.* forwardedindex.2.whitelist = (_audit|_internal|_introspection|_telemetry|_metrics|_metrics_rollup) forwardedindex.filter.disable = false indexAndForward = false
Here is what I have done outputs.conf
[tcpout] defaultGroup=noforward disabled=false [indexAndForward] index=true selectiveIndexing=true [tcpout:forwarders] server:<forwarders>:9997
Below is my props.conf
[default] TRANSFORMS-forwardit = forwardit [host::*.foo.splunk.com] TRANSFORMS-routing = indexing
Below is transforms.conf
[forwardit] REGEX = . DEST_KEY = _TCP_ROUTING FORMAT = forwarders [indexing] REGEX = . DEST_KEY = _INDEX_AND_FORWARD_ROUTING FORMAT = local
Essentially all internal indexes should stay within cluster indexes but rest of index or logs forwarded to external indexes.
I believe you are looking for below: Note: you can only index _internal logs using this method.
Well. This tells me i have to use inputs.conf to ensure routing. By default I want to forward logs. But if i see internal logs i will index it and not forward it. This basically is telling me i have to put _INDEX_AND_FORWARD_ROUTING on all internal inputs.conf this can cause the issue.
For me by default i want to forward new indexes created and internal indexes has to be indexed locally. My thoughts is , setup tcpgroup for forwarders and in outputs.conf and inputs.conf i should modify but not sure how.
The Splunk Doc is very much detailed on the question you have asked. check it out using below link.