topic Re: What are the steps to install Monitoring Console on the License Master instance? in Monitoring Splunk

What are the steps to install Monitoring Console on the License Master instance?

abhi04 — Sat, 06 Jun 2020 00:22:20 GMT

We are building the splunk clustered environment for dev environment. We have a License Master setup. We want the Monitoring console to be setup on the same License master instance. Please let me know what are the steps to be followed for that?

Thanks in advance.

Re: What are the steps to install Monitoring Console on the License Master instance?

richgalloway — Tue, 03 Dec 2019 19:30:00 GMT

The process is the same no matter where you install the MC. See https://docs.splunk.com/Documentation/Splunk/8.0.0/DMC/Configureindistributedmode for the instructions.

Re: What are the steps to install Monitoring Console on the License Master instance?

abhi04 — Tue, 03 Dec 2019 19:36:01 GMT

@richgalloway , thanks for the quick reply. So, I just need to login to the License Master GUI and follow the steps? Just want to make sure.

Re: What are the steps to install Monitoring Console on the License Master instance?

richgalloway — Tue, 03 Dec 2019 20:08:39 GMT

That is correct. You will need credentials for all of the other instances so you can add them as search peers.

Re: What are the steps to install Monitoring Console on the License Master instance?

woodcock — Wed, 04 Dec 2019 07:20:23 GMT

Go to Settings -> Search peers and ensure that ALL Splunk infrastructure nodes are peers. When you peer the Cluster Master, the Indexers should peer in, but if not, add those, too.
Go to Monitoring Console -> Setup -> General Setup and select Distributed Mode then edit each peer to manually assign the correct roles. Click Apply and then PROFIT!!!

Also see here:
https://answers.splunk.com/answers/702341/turn-on-monitoring-console-distributed-mode-via-cl.html

Re: What are the steps to install Monitoring Console on the License Master instance?

straitsresearch — Wed, 04 Dec 2019 07:25:04 GMT

I hope this could help you out.

Most Common Implementation and Deployment Framework

Install splunk enterprise on license master and configure license master
Install splunk enterprise on indexers and configure indexers (alternatively for indexer cluster, install Splunk enterprise on cluster master and indexers and configure indexer cluster)
Install splunk enterprise on search heads and configure search heads
Install splunk enterprise on deployment server and configure deployment server
Install splunk universal forwarder on input devices and configure universal forwarders to connect to deployment server and to forward to indexers
Install Splunk enterprise on DMC monitoring console server and configure monitoring console
(Optional) – Install Splunk enterprise on heavy forwarders and configure heavy forwarders

Install and Configure Splunk Indexer

Install Splunk Enterprise on Linux Server (If you need to create a Linux Server first, visit ___)
Configure Splunk Instance to be an Indexer
Connect Splunk Indexer to Splunk Search Head (Must Configure Search Headfirst, see instructions here)
Peer Splunk Indexer to DMC (Monitoring Console) for monitoring

Re: What are the steps to install Monitoring Console on the License Master instance?

abhi04 — Thu, 05 Dec 2019 11:13:18 GMT

@woodcock , but the monitoring console is to be shared with License master. The clustered indexers cannot be added.For monitoring an indexer cluster and you are hosting the monitoring console on an instance other than the cluster master, you must add the cluster master as a search peer and you must configure the monitoring console instance as a search-head in that cluster.

So, I believe in my case the License master needs to be added as a search head cluster as the DMC needs to be configured in this same instance.
So, Can the License master be added as a search head cluster?

Re: What are the steps to install Monitoring Console on the License Master instance?

woodcock — Thu, 05 Dec 2019 15:24:11 GMT

You are mixing up concepts and terms. There is no such thing as a management console so I have no idea what you mean there. A License Master already IS a Search Head, it just doesn't have any peers by default and you need to change that in order for it to also become the Monitoring Console. I have done this many times. Just add the peers (either directly, or via the Cluster Master) and run the setup.

Re: What are the steps to install Monitoring Console on the License Master instance?

abhi04 — Thu, 05 Dec 2019 15:32:58 GMT

I am sorry, just corrected the "management" to "monitoring". What I meant is that the clustered indexers cannot be added to the search peers directly in the splunk instance web where monioring console needs to be setup. The cluster master needs to be added as a search peer in the monitoring console. Please correct me if I am wrong here.

Re: What are the steps to install Monitoring Console on the License Master instance?

woodcock — Thu, 05 Dec 2019 15:41:16 GMT

You can do either; it is your choice. Personally, I don't trust the Cluster Master and I directly peer. Old habits die hard but you do you.

Re: What are the steps to install Monitoring Console on the License Master instance?

abhi04 — Thu, 05 Dec 2019 15:52:35 GMT

Ok, so we can also add indexers which are clustered directly as search peers individually?

Re: What are the steps to install Monitoring Console on the License Master instance?

woodcock — Thu, 05 Dec 2019 16:11:49 GMT

Yes, that way if you have an event where your splunk servers bounce and the Cluster Master does not come back, you the Monitoring Console will still see the Indexers.

Re: What are the steps to install Monitoring Console on the License Master instance?

abhi04 — Thu, 05 Dec 2019 18:09:14 GMT

@woodcock Thanks for the response 🙂 I have added the instances individually and can see the data for those instances in Monitoring Console now. I will be greatful If you could give an insight on the below doubts as well.

What server roles needs to assigned to each instance. The KV roles are to be set only for the search head?
While Applying Changes I got an error message for one of the search heads that said "Atleast one of the instance is not forwarding its internal logs". But I do see the data and graphs for the servers in the Monitoring Console.

Re: What are the steps to install Monitoring Console on the License Master instance?

woodcock — Thu, 05 Dec 2019 18:17:32 GMT

Search Heads should get Search Head and KV Store, everything else should be obvious. You probably have Search Heads that do not have outputs.conf to send their logs to the Indexers which is the warning and you should fix that. Yes, EVERY node should be added as a search peer. Even your Heavy Forwarders which should be set as Indexer.