topic Re: Can I Add selected heavy forwrders in splunk monitoring console in forwarder section? in Monitoring Splunk

Can I Add selected heavy forwrders in splunk monitoring console in forwarder section?

splbsm — Tue, 12 Feb 2019 13:00:08 GMT

HI,

I want to be able to add only a few selected heavy forwarders in my distributed monitoring console.
so basically I want to use wildcard (or may be a text file with list of forwarders or something similar ) for hostnames of these HFs and only add these matching HF's in my Monitoring console in forwarder section..
Is this possible in splunk ?

Regards

Re: Can I Add selected heavy forwrders in splunk monitoring console in forwarder section?

chrisyounger — Tue, 12 Feb 2019 19:15:13 GMT

Hi @splbsm

If you are a customer that has a few heavy forwarders then it probably means you are large enough that you should consider having a stand-alone monitoring console. With a stand-alone monitoring console you should only add the heavy forwarders you care about as search peers. This way your less important heavy forwarders won't be displayed in the monitoring console. Once a server is defined as a search peer to the monitoring console it will be displayed. You can't filter it using a wildcard.

All the best.

Re: Can I Add selected heavy forwrders in splunk monitoring console in forwarder section?

splbsm — Wed, 13 Feb 2019 09:02:00 GMT

Thanks for taking time for this post Chris, much appreciated.

Yes, at the moment I already have these HF's added as indexers.
But I'd like to add and see these HF's as HF in monitoring console in the forwarder section.
as you already guessed, I can NOT add all UF and HFs there because I have 1000s of UF's.

So Can I not add only a few select HF's in as HFs in monitoring console under forwarders?

Regards

Re: Can I Add selected heavy forwrders in splunk monitoring console in forwarder section?

splbsm — Wed, 13 Feb 2019 11:56:33 GMT

Any other Idea please? anyone?

Re: Can I Add selected heavy forwrders in splunk monitoring console in forwarder section?

splbsm — Wed, 13 Feb 2019 12:13:42 GMT

Any Other Idea please?

Re: Can I Add selected heavy forwrders in splunk monitoring console in forwarder section?

DMohn — Tue, 29 Sep 2020 23:13:25 GMT

You cannot select to have only a few forwarders in the Monitoring Console, as this is depending on having the "full view" of everything going on in your environment.

But you could go for the following solution:

Add all forwarders to the MC
Use the dmc_forwarder_assets alert
limit the search results to the HFs relevant for you by creating a lookup file

A search could look like this:

| inputlookup dmc_forwarder_assets
  [|inputlookup your_hf_list.csv | return hostname]
| search status="missing"

This should give you a list of all missing HF out of your selection.

Re: Can I Add selected heavy forwrders in splunk monitoring console in forwarder section?

splbsm — Wed, 13 Feb 2019 14:15:04 GMT

You mean "DMC Forwarder - Build Asset Table " Alert please?

Re: Can I Add selected heavy forwrders in splunk monitoring console in forwarder section?

splbsm — Wed, 13 Feb 2019 14:23:15 GMT

Also, I have some 60k UF's. if I add them all , will it not be a huge risk for my MC performance? Please advice.

Re: Can I Add selected heavy forwrders in splunk monitoring console in forwarder section?

DMohn — Wed, 13 Feb 2019 14:41:17 GMT

Start by enableing the forwarder monitoring in the MC with Settings => Forwarder Monitoring Setup. You can reduce the data colletion interval if you desire.

This will enable the MC to run internal saved searches, one of which builds the forwarder asset table. This can be accessed in a regular Splunk search with | inputlookup dmc_forwarder_assets. From there on you can build your custom alert which will only cover your selected Heavy Forwarders. This is not a built-in MC alert anymore, as the standard alerts will alert for any missing forwarder. So your should leave these alerts turned off.

And no, this will not be a huge performance risk for your instance if you have sized it accordingly. Be aware that a MC used for such a large architecture has to be a standalone instance, with no other funcionalities.

Re: Can I Add selected heavy forwrders in splunk monitoring console in forwarder section?

splbsm — Wed, 13 Feb 2019 14:50:06 GMT

yes, it is standalone MC. thanks..

Re: Can I Add selected heavy forwrders in splunk monitoring console in forwarder section?

DMohn — Thu, 14 Feb 2019 08:26:54 GMT

If this works for you, could you please mark the answer as accepted, so others will so that there's a solution? Thanks 😉