Why does the Monitoring Console not update with new indexer information?

[kairisrj]

I've replaced 2 or my 4 indexers in the cluster on an AWS deployment. The Indexer Cluster Status shows the right list of indexers but the main monitoring console overview shows the original 4 with 2 as down. How can I get the monitoring console to refresh so it sees the right instances?

[jhaveko]

I am also here from the future to say that this still works in 2024!

[lguinn2 · Accepted solution]

I've had old servers remain in the Monitoring Console before. If the Indexer Clustering dashboard on the cluster master is correct, then try this: simply go to the Settings tab in the Monitoring Console and choose General Setup. (Not the overall Settings menu.)
Take a look at the servers listed. If everything looks reasonable, just click Save. You might also do something similar for Forwarder Setup within the MC.

This may clear older entries from the monitoring tables.

[DBattisto]

I'm here from the future to say that this also fixed the issue I had.

I couldn't figure out how to Google this issue. I completely rebuilt and replaced my indexer, and spent about an hour doing a bunch of grep commands on my search head to see if the IP address for the old indexer still remained in one of the config files.

Nope. Just gotta click 'save'. ...how intuitive 🙂

[kairisrj]

That did the trick

[nawazns5038]

I just added a new indexer to the indexer cluster and the cluster master shows the updated list but not the monitoring console. It shows that the state of the new indexer as new . As a result does not have the updated list on the number of instances.

How do I change it to configured ?

[lguinn2]

To update the list of names in the Monitoring Console, you need to go to Monitoring Console >> Settings >> General Setup.
You should see your "new" server in the list. You may need to edit its roles, but you might not need to edit anything.
But ALWAYS click on Apply Changes. This will ensure that the Monitoring Console (MC) updates its configuration to the same settings that you see on the screen. When you enter the General Setup windows, the MC will show you all the servers that it finds, but the new servers (or settings) are not actually configured in the MC until you click Apply Changes.

[cblanton]

+1, my settings were correct, but Apply Changes was apparently missing. fixed it immediately

[nawazns5038]

Yes.. It worked .. Thanks !!!

[lguinn2]

Do the old indexers appear when you look at the Indexer Clustering dashboard on the cluster master? If yes, then that is why the old indexers are still appearing in the monitoring console. The cluster master will not remove the old indexers from all of its internal tables until you run the following command:

splunk remove cluster-peers -peers guid1,guid2

To get the guid of an indexer, go to the Indexer Clustering dashboard and choose the Indexers tab. Use the small arrow to the left of the indexer name to view its details, including its guid. You will definitely want to cut-and-paste this information into the remove cluster-peers command!

[kairisrj]

No, index clustering shows the correct list. I ran the command you suggested, but Monitoring Console is not updated.

[lguinn2]

To update the list of names in the Monitoring Console, you need to go to Monitoring Console >> Settings >> General Setup.
You should see your "new" server in the list. You may need to edit its roles, but you might not need to edit anything.
But ALWAYS click on Apply Changes. This will ensure that the Monitoring Console (MC) updates its configuration to the same settings that you see on the screen. When you enter the General Setup windows, the MC will show you all the servers that it finds, but the new servers (or settings) are not actually configured in the MC until you click Apply Changes.

[sherm77]

I had to edit the ~/etc/apps/splunk_monitoring_console/lookups/assets.csv file to remove a decomm'd indexer, and also remove it from ~/etc/apps/splunk_monitoring_console/local/splunk_monitoring_console_assets.conf. Once I did that, it disappeared from the monitoring console without restart.