Monitoring Splunk

Monitoring Console configuration problem||Indexers (which part of indexer cluster) does appear at MC||mc_auto_config

Wohamed_wakkad
Explorer

"I am writing this thread to summarize the Monitoring Console (MC) configuration and highlight the non-obvious nuances within the configurations and troubleshooting/error sections."

Steps for Implementing the Monitoring Console (MC) in an Indexer Cluster Infrastructure

1. Connect the Monitoring Console to the Cluster Manager

Configure the Monitoring Console as a search head and connect it to the Cluster Manager (formerly Cluster Master).  // if  not configured the indexers doesn't shown at MC dashboard

[clustering]

manager_uri = https://Manager-URI:8089

mode = searchhead

pass4SymmKey = whatever

2. Forward Internal Logs to the Indexers

Ensure all component internal logs (_internal and _introspection) are forwarded to the indexer cluster from all roles (Search Heads, License Masters, Deployment Servers, and Cluster Managers).

Note: Deployment Server (DS) internal logs are stored locally and must be forwarded to the indexers using selective forwarding configuration.

[indexAndForward]
# Strictly disables local indexing on this host
index = false

[tcpout]
defaultGroup = static_indexer_cluster

[tcpout:static_indexer_cluster]
server = <indexer_01_ip>:9997, <indexer_02_ip>:9997, <indexer_03_ip>:9997

for DS:

[indexAndForward]
index = true
selectiveIndexing = true

[tcpout]
# Prevents automatic untargeted forwarding of every input
defaultGroup = none

[tcpout:static_indexer_cluster]
server = <indexer_01_ip>:9997, <indexer_02_ip>:9997, <indexer_03_ip>:9997

 

3. Set Search Head Cluster and Indexer Cluster Labels

Verify and apply distinct cluster labels for both your Search Head Clusters (SHC) and Indexer Clusters. (Note: These are typically defined during the initial cluster initialization).

4. Add All Component Instances as Search Peers

Add all non-indexer instances (Search Heads, License Masters, Deployment Servers, and Cluster Managers) to the Monitoring Console as search peers. Do not manually add individual indexers that are already members of the managed indexer cluster, as they are discovered automatically via the Cluster Manager.

from settings>Distributed search

search peer and then add them

5. Configure the Monitoring Console in Distributed Mode

Set up and enable the Monitoring Console in distributed mode as outlined below.

 from settings>Monitoring console

settings>general settings

ensure each machine has its role.

the search head roles are search head and KV store.

For more clarification about the above step look at the screenshots bellow:

Wohamed_wakkad_0-1781357126015.png

 

the final expected output:

Wohamed_wakkad_1-1781357233107.png

finally for the bellow section review @kml_uvce  comment

enable automatic distributed mode configuration:

  1. Edit the splunk_monitoring_console_assets.conf file in etc/apps/splunk_monitoring_console/local.
  2. Under the settings stanza, set mc_auto_config to enable, as shown:
     

    [settings]

    mc_auto_config = enabled

     

    because when check the configuration it cause an warning :                                                                                                                                                                                                                                                                                                 Invalid key in stanza [settings] in /opt/splunk/etc/apps/splunk_monitoring_console/local/splunk_monitoring_console_assets.conf, line 2: mc_auto_config (value: enabled).
Labels (1)
0 Karma

kml_uvce
Builder

On the last item, the mc_auto_config section is actually fine as written. The value enabled is correct — it's exactly what the official docs show:

 
 
[settings]mc_auto_config = enabled

https://help.splunk.com/en/splunk-enterprise/administer/monitor/10.4/configure-the-monitoring-consol...

The message you're hitting —

Invalid key in stanza [settings] … mc_auto_config (value: enabled)

— is a conf-validation warning, not a failure. That "Invalid key in stanza" wording fires when the key isn't found in the matching .conf.spec file. In several builds mc_auto_config isn't listed in splunk_monitoring_console_assets.conf.spec, so btool/conf validation flags it cosmetically even though splunkd honors it at runtime — same class as the old python.version warning that can be safely ignored.

So the config isn't broken. To confirm it's actually applying, don't go by the warning — check whether the MC auto-populates in distributed mode (Settings > General Setup, asset table). Per the docs, with mc_auto_config = enabled it takes effect on restart, or automatically every hour with no restart required.

kamal singh bisht
0 Karma

Wohamed_wakkad
Explorer

thx i refer to your reply at the base topic

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Automated Threat Analysis: Available in ES Premier

Automated Threat Analysis: Centralize and Accelerate Phishing Investigations in Splunk Enterprise ...

What’s New in Splunk AI: Volume 02

Welcome to the second edition of “What’s New in Splunk AI” where we look at the latest and greatest updates, ...

Best Practices: Splunk auto adjust pipeline queue

When you enable autoAdjustQueue in Splunk, maxSize should be understood as the queue size Splunk starts with ...