Are there alternative ways to monitor forwarders?

[satkan100]

My splunk environment we have not enable forward management so for me difficult to manage the forwarder host up & down status .

If possible to monitor any other methods? Example App or query if anyone knows please share.

[aakwah]

/opt/splunk/var/log/splunk/metrics.log contains information about incomming connections from forwarders, by default these events indexed under _internal index.

[adonio · Accepted solution]

hello there,

i guess there are couple different ways to achieve.
the way i approach this is by checking if splunk internal data is flowing. if it does -> all good, if it doesnt -> probably connection error or forwarder is down -> alert and check
here is a quick and dirty way to achieve it
| tstats count as event_count by host where index = _interanl
from there you can take it however you like it, the nice part about it is that |tstats takes into consideration the timepicker.
so you can schedule a report / alert
also, you can create a lookup with list of all forwarders and update it every week / day / hour etc, and then run a search that compare existing forwarders to that list

hope it helps

[satkan100]

Hi

Thanks for the your update.

| tstats count AS event_count WHERE index=_internal by host from this query i am able get the details forwarder details. if any possible to create dashboard from this query forwarder on or off status?

[adonio]

the purpose of the query above is to tell you if a forwarder is not sending internal data, which might indicate that its down.
sure, set your threshold for the time you would like to be alerted on and save this search as a scheduled report.
add the report to a dashboard.
if it answered your question, please mark as answered