Deployment Architecture

Forwarder logs in an indexer cluster setting

asiaque_mBank
Engager

Hi,

I have recently started migrating our enterprise infrastructure to an indexer cluster from a previous standalone Splunk server infrastructure. Before the migration, I could see all the splunkd.logs from all forwarders in the _internal index on the server. After switching to the cluster, I can no longer see them when searching in index=_internal via the search head.
I suspect it is due to the fact that _internal is... internal, so it is not clustered among cluster nodes and therefore not searchable via the cluster search head.

What is the proper way to view them? I have splunk web disabled on cluster peers for security reasons as it was never supposed to be searched directly. It feels idiotic having this log-centralising product and not being able to see its logs in a central place.

Tags (1)
0 Karma

jkat54
SplunkTrust
SplunkTrust

You need a combination of outputs.conf and some settings within. An example is below. The whitelist .* says all indexes, and the the 1.blacklist overrides the already configured settings for blacklisted indexes. This should go into $SPLUNK_HOME/etc/appName/local/outputs.conf

[tcpout:INDEXERS]
server = indexer1:9997,indexer2:9997
compressed = true
autoLB = true
forwardedindex.0.whitelist = .*
forwardedindex.1.blacklist =
forwardedindex.2.whitelist =
0 Karma
Get Updates on the Splunk Community!

Automatic Discovery Part 1: What is Automatic Discovery in Splunk Observability Cloud ...

If you’ve ever deployed a new database cluster, spun up a caching layer, or added a load balancer, you know it ...

Real-Time Fraud Detection: How Splunk Dashboards Protect Financial Institutions

Financial fraud isn't slowing down. If anything, it's getting more sophisticated. Account takeovers, credit ...

Splunk + ThousandEyes: Correlate frontend, app, and network data to troubleshoot ...

 Are you tired of troubleshooting delays caused by siloed frontend, application, and network data? We've got a ...