topic Re: How to get Resource Usage information of Heavy Forwarders in Monitoring Console in Deployment Architecture

How to get Resource Usage information of Heavy Forwarders in Monitoring Console

acavenago — Tue, 20 Feb 2024 10:45:14 GMT

Hello,

I have a multi-site cluster at version 9.0.1, with several Indexers, SHs, and HF/UFs.

The Monitoring Console is configured on the Cluster Manager, and "Forwarder Monitoring" is enabled, which allows me to see the status of the forwarders.

What is missing is the possibility to select HF in the Resource Usage section of the Monitoring Console. They are not available.

How can I get them to appear in Resource Usage in the Monitoring Console?

Thank you,

Andrea

Re: How to get Resource Usage information of Heavy Forwarders in Monitoring Console

kiran_panchavat — Fri, 23 Feb 2024 14:21:58 GMT

@acavenago

Ensure that your HFs are correctly configured and connected to the Splunk environment.

Verify that the HFs are sending data to the indexers and are part of the cluster.

Verify that communication between the indexers and HFs is functioning correctly.

In case you’re setting up for an indexer cluster or search head cluster then you must need to set up a cluster label.

In case of indexer cluster:

Go to the CLI of your master node.

And run this following command:

splunk edit cluster-config -cluster_label <CLUSTER LABEL>

Add search peers:

1. Log in to the instance which you want to set up as a monitoring console (in our case it will be the master node)
2. Go to Setting and Distributed Search. And click on Search Peer.
3. Click on new search peer and add all search head, license master, non-clustered indexers, and clustered search head.
4. Repeat this process several times based on the number of instances you want to add.
5. We don’t need to add a master node here because we are doing all of this stuff into master nodes only. So it will automatically add.
6. Now go to the setting > monitoring console > setting > general setup

7. Click on distributed and continue.
8. Come down and check the status of all remote instances.
9. Check server roles are showing correct roles for that particular instance or not, if not then click on action > edit and edit server roles.
10. Now go to the overview page of your newly set up monitoring console.

Forwarder setup in monitoring console:

First, go to your newly set up monitoring console and click on forwarders and forwarders: instance.

Now click on setup, to configure this page.

Now enable, forwarder monitoring and choose data collection intervals. Then click on save and continue.

Then this process or search will fetch all of your forwarder assets and will build a forwarder management dashboard within the monitoring console by running a scheduled search named “DMC Forwarder – Build Asset Table”.

After doing those above steps you will avail to see all of your forwarder’s information as shown below.

Re: How to get Resource Usage information of Heavy Forwarders in Monitoring Console

acavenago — Mon, 26 Feb 2024 17:17:39 GMT

Hi @kiran_panchavat , thank you for all the information.

I was already able to list HF info in MC/Forwarders menu.

What I need is to have HF also listed in MC/Resource Usage, where right now I have only Cluster Manager and Indexers nodes.

Kind regards,

Andrea

Re: How to get Resource Usage information of Heavy Forwarders in Monitoring Console

PickleRick — Mon, 26 Feb 2024 22:44:01 GMT

HFs have always been a bit of an "ugly duckling". They are forwarders so they are covered by forwarder monitoring but only covering the same set of parameters as UFs.

You can try to add them as indexers to the MC which should give you their health parameters (but can cause issues if you're using forwarder license on them).

Generally there is no single good answer since some of the HFs can't be monitored in any way other than by checking the _internal log (as it is done for UFs) so you can't add them as reachable search peers to MC.