topic Re: Searching internal logs for heavy forwarder in Monitoring Splunk

Searching internal logs for heavy forwarder

nawazns5038 — Mon, 22 Jan 2018 19:47:12 GMT

I am forwarding the logs from the heavy forwarder using the outputs.conf and as a result all the internal logs have been forwarded as well.

Is there a way to search the _internal logs internally in that instance itself. There is a dashboard provided for an Addon for that heavy forwarder .. it cannot run as there are the internal logs cannot be searched .

Re: Searching internal logs for heavy forwarder

horsefez — Mon, 22 Jan 2018 19:54:44 GMT

Hi,
well you might have set the parameter indexAndForward=false in the outputs.conf which will not store a copy of the data on the heavy forwarder and instead only send the data to your indexers.

If you want to do what you are planning you have to set the parameter to true.

[tcpout]
indexAndForward=true

Re: Searching internal logs for heavy forwarder

nawazns5038 — Mon, 22 Jan 2018 20:04:38 GMT

I want only the internal data to be indexed locally. I want all other data that we are collecting using the heavy forwarder to be forwarded to indexers

Re: Searching internal logs for heavy forwarder

nickhills — Mon, 22 Jan 2018 20:21:53 GMT

You will want to configure TCP routing to handle the sourcetypes separately.
One way is to leave indexAndForward=false globally, and re-route your internal logs to add that parameter for the tcpout group.

http://docs.splunk.com/Documentation/SplunkCloud/6.6.3/Forwarding/Routeandfilterdatad

in props.conf

[splunkd] (or if nothing else is generated 'on' the host you could use host? to save doing it for each sourcetype)
TRANSFORMS-routing=internal

in transforms.conf

[internal]
REGEX=.
DEST_KEY=_TCP_ROUTING
FORMAT=internalLogs

in outputs.conf

[tcpout]
defaultGroup=default

[tcpout:internalLogs]
indexAndForward=false
server=your_indexer:9997

[tcpout:default]
server=your_indexer:9997

Another approach is to use forwardedindex which is covered in that doc.

Re: Searching internal logs for heavy forwarder

nawazns5038 — Mon, 22 Jan 2018 21:28:33 GMT

http://docs.splunk.com/Documentation/SplunkCloud/6.6.3/Forwarding/Routeandfilterdatad#Index_one_input_locally_and_then_forward_all_inputs

Did the trick