Monitoring Splunk

Searching internal logs for heavy forwarder

nawazns5038
Builder

I am forwarding the logs from the heavy forwarder using the outputs.conf and as a result all the internal logs have been forwarded as well.

Is there a way to search the _internal logs internally in that instance itself. There is a dashboard provided for an Addon for that heavy forwarder .. it cannot run as there are the internal logs cannot be searched .

0 Karma
1 Solution

nickhills
Ultra Champion

You will want to configure TCP routing to handle the sourcetypes separately.
One way is to leave indexAndForward=false globally, and re-route your internal logs to add that parameter for the tcpout group.

http://docs.splunk.com/Documentation/SplunkCloud/6.6.3/Forwarding/Routeandfilterdatad

in props.conf

[splunkd] (or if nothing else is generated 'on' the host you could use host? to save doing it for each sourcetype)
TRANSFORMS-routing=internal

in transforms.conf

[internal]
REGEX=.
DEST_KEY=_TCP_ROUTING
FORMAT=internalLogs

in outputs.conf

[tcpout]
defaultGroup=default

[tcpout:internalLogs]
indexAndForward=false
server=your_indexer:9997

[tcpout:default]
server=your_indexer:9997

Another approach is to use forwardedindex which is covered in that doc.

If my comment helps, please give it a thumbs up!

horsefez
SplunkTrust
SplunkTrust

Hi,
well you might have set the parameter indexAndForward=false in the outputs.conf which will not store a copy of the data on the heavy forwarder and instead only send the data to your indexers.

If you want to do what you are planning you have to set the parameter to true.

[tcpout]
indexAndForward=true
0 Karma

nawazns5038
Builder

I want only the internal data to be indexed locally. I want all other data that we are collecting using the heavy forwarder to be forwarded to indexers

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...