How to forward ONLY _internal logs from standalone Search Head to Indexer Cluster?

[naqviah]

Can someone guide me on how to forward ONLY _internal logs from a standalone SH (Search Head) to an indexer cluster in a distributed environment?

[gcusello]

Hi naqviah,
you have to configure your Search Head as an Heavy Forwarder, using web interface:

• [Settings -- Forwarding and receiving -- Forwarding defaults] to say to the Search Head to not locally index logs;
• [Settings -- Forwarding and receiving -- Configure forwarding] to say to the Search Head which are Indexers to send logs.

and restart Search Head.

Bye.
Giuseppe

[jonmargulies]

What you want to do is create a local outputs.conf configuration on the Search Head that looks like this:

  [tcpout]
  forwardedindex.0.whitelist = .*
  forwardedindex.1.blacklist = _.*
  forwardedindex.2.whitelist = (_internal)

This will replace the default outputs.conf entry, which sends more internal indexes to indexers. Other than the above, all you'll have to do is set outputs.conf to point at the indexer cluster.

See this old answer for more detail on how the forwardedindex whitelists and blacklists work: https://answers.splunk.com/answers/339930/how-do-forwardedindex-whitelists-and-blacklists-wo.html

[naqviah]

Do i have to configure the inputs.conf on the IDX clusters?

This is what i currently have, but there is a communication error between the SH and the IDX_Cluster:

[indexAndForward]
index = true

[tcpout]
defaultGroup = idx-indexers
forwardedindex.filter.disable = false
indexAndForward = 1

[tcpout]
forwardedindex.0.whitelist = .*
forwardedindex.1.blacklist = _.*
forwardedindex.2.whitelist = (_internal)

[tcpout:idx-indexers]
autoLBFrequency = 40
disabled = 0
server = :9997,:9997:9997
sslCertPath = $SPLUNK_HOME/etc/auth/server.pem
sslPassword =
sslRootCAPath = $SPLUNK_HOME/etc/auth/cacert.pem
sslVerifyServerCert = 0
useACK = 1

Anything im doing wrong here?

[brreeves_splunk]

The SH should only be generating internal logs anyway...unless you want to skip the other internal indexes.

What EXACTLY are you trying to accomplish?