Install Monitoring Console on the Search Head node in distributed mode

[beneteos]

Hello,

We have migrated our standalone installation of Splunk Enterprise to a "Small enterprise distributed deployment".
This is a really small distributed deployment because the load is essentially on indexing capacity, even though it's less than 100Go daily (our licence allows 80Go) and search load is really low.

So we have :
- 1 Search Head
- 2 indexers (no cluster)

The search head also acts as license master and deployment server (just HEC configs and indexes replication to indexers).

Now the question is : Is it possible to install Monitoring Console on the Search Head node ?

We've well seen the recommandation here, and especially :
"When you set up the monitoring console in distributed mode, it creates one search group for each server role, identified cluster, or custom group. Unless you use a "splunk_server_group" or the "splunk_server" option, only search peers that are members of the indexer group are searched by default. Because all searches that run on the monitoring console instance follow this behavior, non-monitoring console searches might have incomplete results."

I'm not sure I really understand this, but as we only have 2 indexers and since they are the nodes that we want to put in the indexer group on the MC side, could it really leads to incomplete searchs ?

It seems that this is the same advice given on dashboard, via the MC general setup page when trying to activate in distributed mode :
"Do not configure the DMC in distributed mode if this is a production search head. Doing so can change the behavior of all searches on this instance. This is dangerous and unsupported."

As already said, load consideration is secondary because we do not have a heavy searching activity.

Thanks a lot.

[beneteos]

Hi @gcusello and thank you for your answer !

We do not use DS for Splunk Forwarders management (we do that with puppet), so we only have 2 clients, our indexers, in order to replicate HEC configs and so get the same tokens on both indexers. Normally we don't need to change these configs except to add new HEC inputs, so really ponctually.

As for the Monitoring Console, we are basically afraid of warnings saying that it could lead to incomplete search results. But as I said, we only have two indexers that are already configured as search peers on the SH, so it's exactly the same pool of indexers that we want to integrate in MC distributed mode.
In terms of load, our search head is used very sparingly, with only a few searches per day. I think there's a maximum of ten connections/searches per day.

So if I understand you correctly, there are no real risks of functionality loss, the question is more about load, right ?

[gcusello]

Hi @beneteos ,

good for you, see next time!

Ciao and happy splunking

Giuseppe

P.S.: Karma Points are appreciated 😉

[gcusello]

Hi @beneteos ,

yes, use a correct reference hardware and monitor resources use.

Ciao.

Giuseppe 

[gcusello · Accepted solution]

Hi @beneteos ,

usually isn't a best practice to have the Deployment Server on the same server of Search Head, this is possible if you haven't a large use of SH and you have less than 50 clients to manage with DS; otherwise you need a dedicated server for DS.

About Monitoring Console, you can use it on the Search Head if you haven't a large use of SH: monitor the resource usage of your SH.

About the configuration, you have to configure all the servers as search peers for the Search Head.

Ciao.

Giuseppe